-- Logs begin at Thu 2022-08-04 03:10:12 UTC, end at Thu 2022-08-04 05:27:11 UTC. -- Aug 04 04:25:00.979387 dv-cinder-851279-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 04 04:25:00.988813 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 04 04:25:00.990022 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Aug 4 04:25:00 2022] *** Aug 04 04:25:00.990022 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 04 04:25:00.990022 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 04 04:25:00.990022 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: nodename: dv-cinder-851279-5-iscsi Aug 04 04:25:00.990022 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: machine: x86_64 Aug 04 04:25:00.990022 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: clock source: unix Aug 04 04:25:00.990022 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: pcre jit disabled Aug 04 04:25:00.990022 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: detected number of CPU cores: 4 Aug 04 04:25:00.990022 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: current working directory: / Aug 04 04:25:00.990022 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: detected binary path: /usr/bin/uwsgi-core Aug 04 04:25:00.990022 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: your processes number limit is 47680 Aug 04 04:25:00.990022 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: your memory page size is 4096 bytes Aug 04 04:25:00.990022 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: detected max file descriptor number: 2048 Aug 04 04:25:00.990022 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: lock engine: pthread robust mutexes Aug 04 04:25:00.993311 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: thunder lock: enabled Aug 04 04:25:00.993311 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 04 04:25:00.993311 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 04 04:25:01.031210 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: Python main interpreter initialized at 0x559d653349c0 Aug 04 04:25:01.031210 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: python threads support enabled Aug 04 04:25:01.031210 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: your server socket listen backlog is limited to 100 connections Aug 04 04:25:01.031210 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: your mercy for graceful operations on workers is 90 seconds Aug 04 04:25:01.031693 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: mapped 403077 bytes (393 KB) for 2 cores Aug 04 04:25:01.031693 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: *** Operational MODE: preforking *** Aug 04 04:25:01.031928 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: *** uWSGI is running in multiple interpreter mode *** Aug 04 04:25:01.031928 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: spawned uWSGI master process (pid: 80361) Aug 04 04:25:01.032172 dv-cinder-851279-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 04 04:25:01.034464 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: spawned uWSGI worker 1 (pid: 80402, cores: 1) Aug 04 04:25:01.034464 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: spawned uWSGI worker 2 (pid: 80403, cores: 1) Aug 04 04:25:01.034464 dv-cinder-851279-5-iscsi devstack@keystone.service[80361]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 04 04:25:06.857233 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 04:25:06.857233 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: from cryptography.utils import int_from_bytes Aug 04 04:25:06.858038 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 04:25:06.858038 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: from cryptography.utils import int_from_bytes Aug 04 04:25:06.863526 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 04:25:06.863526 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: from cryptography.utils import int_from_bytes Aug 04 04:25:06.864044 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 04:25:06.864044 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: from cryptography.utils import int_from_bytes Aug 04 04:25:06.943729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 04 04:25:06.944007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 04 04:25:06.944295 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] command line args: [] {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 04 04:25:06.944715 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 04 04:25:06.945740 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ================================================================================ {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 04 04:25:06.946186 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] admin_token = **** {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.946442 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] config_dir = [] {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.946633 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.946815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] config_source = [] {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.946932 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 04 04:25:06.947056 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] debug = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.947168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 04 04:25:06.947295 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.947546 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] command line args: [] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 04 04:25:06.947546 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 04 04:25:06.947744 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.947846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ================================================================================ {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 04 04:25:06.947957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.948055 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] admin_token = **** {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.948156 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.948256 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] config_dir = [] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.948350 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.948445 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.948557 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] list_limit = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.948653 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] config_source = [] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.948758 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] log_config_append = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.948861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] debug = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.948961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.949061 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.949201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] log_dir = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.949201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] log_file = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.949375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.949375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.949580 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.949697 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.949798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.949892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.949987 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.950079 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] list_limit = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.950182 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.950301 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] log_config_append = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.950417 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.950536 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.950638 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.950726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] log_dir = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.950812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.950902 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] log_file = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.951002 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.951083 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.951168 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.951244 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.951333 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.951422 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.951504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.951579 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.951659 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.951741 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.951830 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.951915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.952374 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.952494 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.952594 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.952702 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.952890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.952890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.953438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.953595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.953733 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] publish_errors = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.953847 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.953963 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.954133 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.954237 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.954237 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.954514 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.954619 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.954786 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.954891 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.954891 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] standard_threads = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.955162 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.955281 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.955407 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] publish_errors = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.955548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.955677 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.955800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.955924 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.956048 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] use_journal = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.956172 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.956365 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] use_json = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.956490 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.956607 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] use_stderr = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.956607 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] use_syslog = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.956872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.956997 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.957113 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] standard_threads = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.957224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.957336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.957568 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.957677 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.957677 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.958048 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.958228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] use_journal = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.958345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.958458 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] use_json = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.958458 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] use_stderr = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.958674 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.958791 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] use_syslog = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.958911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.959016 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 04:25:06.959123 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.959239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.959343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.959464 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.959569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.959676 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.959781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.959900 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.960015 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.960136 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.960250 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.960369 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.961198 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.961198 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.961411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.961411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.961688 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.961802 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.961921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.961921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.962167 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.962313 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.962429 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.962429 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.962686 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.962799 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.962914 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.962914 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.963198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.963198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.963400 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.963508 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.963625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.963743 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.963847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.963956 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.964117 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.964223 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.964345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.964462 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.964577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.964681 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.964789 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.964897 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.965016 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.965125 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.965241 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.965346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.965451 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.965618 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.965748 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.965857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.965965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.966084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.966189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.966297 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.966422 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.966527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.966633 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.966742 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.966849 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.966956 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.967076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.967182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.967289 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.967412 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.967516 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.967623 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.967748 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.967854 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.967960 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.968067 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.968173 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.968281 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.968469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.968574 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.968692 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.968797 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.968916 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.969024 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.969151 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.969255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.969362 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.969470 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.969595 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.969752 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.969881 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.969996 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.970106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.970226 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.970334 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.970445 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.970607 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.970757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.970909 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.971061 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.971220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.971375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.971540 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.971697 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.971845 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.971996 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.972159 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.972354 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.972494 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.972665 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.972822 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.972938 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.973106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.973224 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.973391 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.973497 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.973699 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.973826 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.974000 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.974119 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.974287 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.974405 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.974577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.974745 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.974923 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.975034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.975201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.975321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.975486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.975593 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.975723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.975846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.975975 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.976110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.976224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.976348 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.976474 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.976588 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.976710 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.976834 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.976953 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.977073 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.977191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.977311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.977430 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.977565 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.977683 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.977794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.977903 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.978027 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.978154 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.978272 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.978379 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.978487 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.978606 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.978714 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.978827 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.978951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.979059 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.979167 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.979275 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.979393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.979500 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.979626 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.979741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.979741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.979951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.980057 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.980176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.980294 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.980401 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.980511 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.980619 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.980727 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.980872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.981056 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.981163 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.981279 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.981399 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.981507 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.981642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.981776 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.981894 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.982012 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.982137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.982249 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.982357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.982466 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.982586 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.982691 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.982812 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.982935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.983042 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.983149 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.983269 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.983375 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.983486 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.983594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.983703 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.983810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] auth.external = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.983936 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.984042 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.984151 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.984151 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.984364 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.984364 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.984591 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.984695 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] auth.password = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.984811 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.984917 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] auth.token = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.985026 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.985166 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.985283 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.985390 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.985512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] auth.external = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.985684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.985792 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.985901 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.986017 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.986136 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.986256 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.986362 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.986473 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] auth.password = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.986582 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.986692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] auth.token = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.986802 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] credential.caching = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.986924 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.987031 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.987140 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.987249 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.987359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.987474 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.987595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.987702 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.987812 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.987922 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.988031 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.988139 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.988261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.988368 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.988477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] credential.caching = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.988586 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.988695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.988804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.988937 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.989057 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.989164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.989274 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.989384 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.989492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.989635 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.989635 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.989854 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.989854 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.990057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.990166 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.990287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.990396 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] federation.caching = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.990507 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.990621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.990731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.990840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.990962 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.991069 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.991177 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.991286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.991396 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.991521 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.991640 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.992302 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.992435 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.992562 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.992673 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] federation.caching = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.992796 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.992923 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.993033 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.993143 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.993261 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.993379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.993487 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.993670 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.993778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] identity.caching = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.993892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.994003 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.994114 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.994245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.994372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.994486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.994597 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.994722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.994835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.995031 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.995140 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.995259 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.995371 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.995371 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.caching = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.995584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.995584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.995805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.995922 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.995922 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.996133 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.996246 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.996373 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.996481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.996596 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.996710 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.996823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.996936 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.997061 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.997170 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.997284 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.997398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.997517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.997652 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.997791 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.997791 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.997997 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.998112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.998229 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.998348 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.998477 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.998592 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.998707 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.998823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.998939 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.998939 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.999176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.999290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.999407 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.999524 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.999641 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.999765 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:06.999890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.000003 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.000129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.000242 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.000360 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.000476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.000608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.000724 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.000842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.000959 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.001074 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.001190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.001344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.001453 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.001609 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.001722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.001839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.001954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.002080 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.002192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.002313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.002428 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.002544 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.002659 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.002785 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.002897 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.003013 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.003127 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.003244 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.003358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.003485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.003596 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.003713 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.003827 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.003947 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.004060 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.004190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.004301 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.004538 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.004654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.004763 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.004875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.005005 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.005124 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.005240 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.005356 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.005473 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.005685 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.005799 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.005916 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.user = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.006032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.006147 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.006263 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.006391 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.006504 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.006619 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.006745 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.006862 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.006974 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.007097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.007224 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.007875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.007994 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.008108 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.008222 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.008345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.008464 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.008573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.008680 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.008787 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.008899 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.009027 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.009405 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.009517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.009706 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.009815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.009935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.010075 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.010185 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.010303 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.010420 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.010420 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.010637 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.010765 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.010880 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.010997 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.011113 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.011229 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.011352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.011481 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.011481 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.011687 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.011802 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.011918 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.012034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.012161 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.012277 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.012393 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.012507 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.012623 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.012740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.012869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.012990 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.013106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.013223 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.013339 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.013456 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.013606 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.013726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.013841 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.013958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.014072 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.014198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.014324 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.014435 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.014545 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.014667 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.014783 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.014900 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.015038 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.015151 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.015269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.015385 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.015502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.015620 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.015749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] resource.caching = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.015864 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.015981 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.016098 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.016215 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.016336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.016464 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.016580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.016699 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.016814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.016932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.017048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.017178 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.017293 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] resource.caching = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.017411 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.017527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.017705 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.017821 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.017957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.018066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.018181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] role.caching = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.018294 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.018409 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] role.driver = sql {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.018531 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.018722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.018830 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.018945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.019059 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.019173 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.019288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.019413 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.019526 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.019645 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.019758 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] role.caching = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.019873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.019999 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] role.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.020110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.020226 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.020340 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.020454 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.020567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.020692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.020803 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.020917 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.021031 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.021151 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.021263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.021390 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.021500 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.021634 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.021751 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.021864 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.021981 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.022112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.022222 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.022339 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.022455 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.022572 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.022711 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.022832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.022966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.023107 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.023733 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.023848 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.023964 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.024092 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.024204 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.024320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.024434 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.024548 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.024659 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.024789 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.024912 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.024912 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.025122 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.025235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.025363 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.025477 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.025732 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.025851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.025970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.026086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.026219 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.026328 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.026445 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.026566 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.026680 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.026795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.026922 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.027041 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] token.caching = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.027156 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.027272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.027387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.027387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.027614 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.027724 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.027847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.027961 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.028081 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.028195 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.028323 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.028435 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.028549 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.028665 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] token.caching = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.028780 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.028936 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.029068 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.029181 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.029300 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.029418 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.029533 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.029669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.029809 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.029809 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.030011 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.030126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.030244 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.030360 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.030489 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.030617 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.030733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.030851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.030966 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.030966 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.031187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.031299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.031413 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] database.connection = **** {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.031539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.031665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.031780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.031898 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.032027 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.032143 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.032260 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.032375 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.032491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.032605 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.032605 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.032835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.032950 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.033072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.connection = **** {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.033201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.033326 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.033442 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.033628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.033756 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.033869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.033984 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.034100 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.034214 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.034331 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.034458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.034570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.034677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.034805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.034921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.035039 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.035168 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.035281 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.035399 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.035529 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.035643 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.035761 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.035887 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 04 04:25:07.036003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.036003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.036003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.036003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.036003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.036003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.036003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 04:25:07.036003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 04 04:25:07.063473 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.063683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.063841 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.063960 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.064086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.064209 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.072397 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.073071 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.103001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.103212 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.104151 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.104367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.158725 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.163472 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.211765 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.211986 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.214833 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.215064 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.277959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.278174 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.278398 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.278619 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.278836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.279034 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.279150 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.279265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.279477 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.279696 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.279913 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.280127 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.303284 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.303491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.303627 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.303754 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.303904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.304022 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.304204 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.304324 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.304455 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.304582 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.304712 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.304712 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.305032 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.305154 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.315942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.316232 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.329076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.329345 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.329575 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.329700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.329855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.329970 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.330089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.330209 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.330328 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.330461 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.330575 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.330694 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.330694 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.330923 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 04:25:07.347767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.347974 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 04:25:07.348133 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.348310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.348424 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 04:25:07.348549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 04:25:07.348692 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.348808 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.348929 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 04:25:07.349049 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.349170 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.349309 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.349425 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.349580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.349698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.349834 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.360144 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.360420 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.360685 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.360801 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.360960 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.361076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.361253 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.361533 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.368501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.369537 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.373289 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.373594 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.373879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.374157 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.374345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.374459 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.374626 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.374759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.374904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.375028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.375179 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.375291 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.375445 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.375557 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.375696 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.375807 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.375958 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.376083 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.376221 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.376491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.376761 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.377024 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.399420 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.400906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.401184 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.401818 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.403321 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.403597 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.405831 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.406107 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.406383 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.408222 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.408497 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.408819 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.412965 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.413257 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.415973 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.416249 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.417526 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.417839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.418125 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.418416 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.418695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.418971 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.419243 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.419515 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.420528 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.420813 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.421089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.421361 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.421657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.421941 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.422212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.422736 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.436569 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.436845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.439636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.439917 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.440783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.443856 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.443998 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.444229 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.447687 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.448037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.449288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.452530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.453075 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.455733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.456031 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.456315 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.456571 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.456657 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.456866 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.457142 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.460007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.460368 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.460824 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.461188 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.461524 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.461900 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.473346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.473655 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.473922 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.474191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.478489 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.478827 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.479135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.479451 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.481261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.482933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.483206 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.486051 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.486330 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.486504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.486629 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.486884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.487158 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.487919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.488189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.491051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.491329 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.491599 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.492444 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.492715 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.499093 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.499370 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.499650 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.499926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.500200 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.500479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.500754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.505232 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.505232 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.505232 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.506478 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.506478 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.506478 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.506478 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.517361 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.520371 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.522562 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.523400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.525630 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.526008 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.527553 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.527835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.528121 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.529100 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.531826 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.533356 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.533668 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.533951 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.534609 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.536199 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.539247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.539535 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.539815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.540087 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.540416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.542095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.545127 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.545405 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.545771 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.546043 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.548386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.548689 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.548965 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.554973 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.555404 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.555847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.556431 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.556708 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.556987 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.557256 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.557521 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.557810 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.558078 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.558351 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.558727 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.558996 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.559315 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.559560 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.559818 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.560072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.562813 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 04:25:07.563084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.563355 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.563619 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.563880 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.564148 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.564409 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.564672 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.565048 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.565309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.565592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.565862 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.566114 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.566372 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 04:25:07.593892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559d653349c0 pid: 80402 (default app) Aug 04 04:25:07.597306 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225f343-0a96-4d77-80f9-dbb98e95cc44 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:07.597614 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225f343-0a96-4d77-80f9-dbb98e95cc44 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:07.597892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225f343-0a96-4d77-80f9-dbb98e95cc44 None None] PATH_INFO: `/v3` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:07.600683 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559d653349c0 pid: 80403 (default app) Aug 04 04:25:07.602775 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1/1] 10.222.0.47 () {50 vars in 791 bytes} [Thu Aug 4 04:25:07 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:15.499091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf4fdd-9478-427b-b2fb-4dd1a010768d None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:15.499747 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf4fdd-9478-427b-b2fb-4dd1a010768d None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:15.499747 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf4fdd-9478-427b-b2fb-4dd1a010768d None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:15.504064 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1/2] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:15.510699 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1dc1de-6321-4a28-a0d3-f30dde9e77c4 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:15.511161 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1dc1de-6321-4a28-a0d3-f30dde9e77c4 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:15.511463 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1dc1de-6321-4a28-a0d3-f30dde9e77c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:15.586512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: INFO dbcounter [None req-8c1dc1de-6321-4a28-a0d3-f30dde9e77c4 None None] Registered counter for database keystone Aug 04 04:25:15.606665 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG oslo_db.sqlalchemy.engines [None req-8c1dc1de-6321-4a28-a0d3-f30dde9e77c4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80402) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 04 04:25:15.612869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writer thread running {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 04 04:25:15.796139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG passlib.handlers.bcrypt [None req-8c1dc1de-6321-4a28-a0d3-f30dde9e77c4 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80402) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 04 04:25:15.796555 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG passlib.handlers.bcrypt [None req-8c1dc1de-6321-4a28-a0d3-f30dde9e77c4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80402) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 04 04:25:15.852996 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-8c1dc1de-6321-4a28-a0d3-f30dde9e77c4 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:15.854641 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8c1dc1de-6321-4a28-a0d3-f30dde9e77c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:15.990330 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 2/3] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 04 04:25:16.115013 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf4fdd-9478-427b-b2fb-4dd1a010768d None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:16.185137 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: INFO dbcounter [None req-0daf4fdd-9478-427b-b2fb-4dd1a010768d None None] Registered counter for database keystone Aug 04 04:25:16.206621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG oslo_db.sqlalchemy.engines [None req-0daf4fdd-9478-427b-b2fb-4dd1a010768d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80403) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 04 04:25:16.213732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writer thread running {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 04 04:25:16.334463 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf4fdd-9478-427b-b2fb-4dd1a010768d None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:16.339846 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1bbf26-fc9a-413c-9596-fb66722ae405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:16.340496 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bbf26-fc9a-413c-9596-fb66722ae405 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:16.340778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bbf26-fc9a-413c-9596-fb66722ae405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:16.341048 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bbf26-fc9a-413c-9596-fb66722ae405 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:16.349792 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1bbf26-fc9a-413c-9596-fb66722ae405 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:16.403125 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8f1bbf26-fc9a-413c-9596-fb66722ae405 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 04 04:25:16.403379 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1bbf26-fc9a-413c-9596-fb66722ae405 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:16.414172 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 2/4] 10.222.0.47 () {58 vars in 1324 bytes} [Thu Aug 4 04:25:16 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 04:25:16.420304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1dc1de-6321-4a28-a0d3-f30dde9e77c4 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:16.435646 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1dc1de-6321-4a28-a0d3-f30dde9e77c4 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:16.440119 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377cde1-9269-4ef5-851b-f7df3f9ba47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:16.441273 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377cde1-9269-4ef5-851b-f7df3f9ba47b admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:16.441574 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377cde1-9269-4ef5-851b-f7df3f9ba47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:16.441844 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377cde1-9269-4ef5-851b-f7df3f9ba47b admin admin] PATH_INFO: `/v3/projects/312c7b7a94774ca2ba3533f4d13a2124` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:16.443357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d377cde1-9269-4ef5-851b-f7df3f9ba47b admin admin] RBAC: Authorizing `identity:get_project(project_id=312c7b7a94774ca2ba3533f4d13a2124)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:16.495248 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d377cde1-9269-4ef5-851b-f7df3f9ba47b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 04 04:25:16.495594 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d377cde1-9269-4ef5-851b-f7df3f9ba47b admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:16.500464 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 3/5] 10.222.0.47 () {56 vars in 1207 bytes} [Thu Aug 4 04:25:16 2022] GET /identity/v3/projects/312c7b7a94774ca2ba3533f4d13a2124 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:25:18.543774 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c2fd3-9acf-44e7-8b51-030e2601fc39 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:18.544319 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c2fd3-9acf-44e7-8b51-030e2601fc39 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:18.544319 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c2fd3-9acf-44e7-8b51-030e2601fc39 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:18.548666 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 3/6] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:18.555358 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e95776-88bb-4077-897a-5538d0c4bf05 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:18.555824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e95776-88bb-4077-897a-5538d0c4bf05 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:18.555958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e95776-88bb-4077-897a-5538d0c4bf05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:18.576571 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-74e95776-88bb-4077-897a-5538d0c4bf05 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:18.578098 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-74e95776-88bb-4077-897a-5538d0c4bf05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:18.586528 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 4/7] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 04 04:25:18.608067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c2fd3-9acf-44e7-8b51-030e2601fc39 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:18.621027 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c2fd3-9acf-44e7-8b51-030e2601fc39 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:18.625188 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b7786-6b75-4486-9f18-93aad87d2d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:18.625823 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b7786-6b75-4486-9f18-93aad87d2d92 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:18.626105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b7786-6b75-4486-9f18-93aad87d2d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:18.626378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b7786-6b75-4486-9f18-93aad87d2d92 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:18.635418 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b7786-6b75-4486-9f18-93aad87d2d92 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:18.637188 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b7786-6b75-4486-9f18-93aad87d2d92 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:18.646236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 4/8] 10.222.0.47 () {58 vars in 1324 bytes} [Thu Aug 4 04:25:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 04:25:18.652207 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e95776-88bb-4077-897a-5538d0c4bf05 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:18.661271 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e95776-88bb-4077-897a-5538d0c4bf05 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:18.666537 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c9910d-e3ca-4fea-8944-d16b5b6b4691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:18.667174 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9910d-e3ca-4fea-8944-d16b5b6b4691 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:18.667456 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9910d-e3ca-4fea-8944-d16b5b6b4691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:18.667732 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9910d-e3ca-4fea-8944-d16b5b6b4691 admin admin] PATH_INFO: `/v3/users/fa79725fac2c4a528df2b060ede4922c` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:18.669102 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c9910d-e3ca-4fea-8944-d16b5b6b4691 admin admin] RBAC: Authorizing `identity:get_user(user_id=fa79725fac2c4a528df2b060ede4922c)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:18.670950 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c9910d-e3ca-4fea-8944-d16b5b6b4691 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:18.672772 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 5/9] 10.222.0.47 () {56 vars in 1198 bytes} [Thu Aug 4 04:25:18 2022] GET /identity/v3/users/fa79725fac2c4a528df2b060ede4922c => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:20.793235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113314fd-d94b-428f-ac7c-772954a80620 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:20.793901 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113314fd-d94b-428f-ac7c-772954a80620 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:20.794031 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113314fd-d94b-428f-ac7c-772954a80620 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:20.795437 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 5/10] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:25:20.801310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29bac6-4662-4022-887f-8d15b78b90d5 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:20.802029 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29bac6-4662-4022-887f-8d15b78b90d5 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:20.802260 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29bac6-4662-4022-887f-8d15b78b90d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:20.822578 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5526590-f873-40fc-a332-840f0268008d None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:20.822906 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5526590-f873-40fc-a332-840f0268008d None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:20.822999 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-da29bac6-4662-4022-887f-8d15b78b90d5 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:20.823182 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5526590-f873-40fc-a332-840f0268008d None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:20.824298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-da29bac6-4662-4022-887f-8d15b78b90d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:20.824658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 6/11] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:25:20.830198 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c852d-1042-4ce1-9944-c4f07962abd2 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:20.830479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c852d-1042-4ce1-9944-c4f07962abd2 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:20.830751 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c852d-1042-4ce1-9944-c4f07962abd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:20.875158 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG passlib.handlers.bcrypt [None req-954c852d-1042-4ce1-9944-c4f07962abd2 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80403) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 04 04:25:20.875574 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG passlib.handlers.bcrypt [None req-954c852d-1042-4ce1-9944-c4f07962abd2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80403) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 04 04:25:20.912127 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 6/12] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:25:20.922074 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-954c852d-1042-4ce1-9944-c4f07962abd2 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:20.923934 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-954c852d-1042-4ce1-9944-c4f07962abd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:20.935404 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29bac6-4662-4022-887f-8d15b78b90d5 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:20.966212 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:21.044625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 7/13] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:25:21.055005 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29bac6-4662-4022-887f-8d15b78b90d5 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:21.073250 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c852d-1042-4ce1-9944-c4f07962abd2 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:21.092380 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302bbe6-ba1d-405f-962e-2f55d60a33f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:21.093083 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302bbe6-ba1d-405f-962e-2f55d60a33f9 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:21.093511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302bbe6-ba1d-405f-962e-2f55d60a33f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:21.093887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302bbe6-ba1d-405f-962e-2f55d60a33f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:21.166625 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e302bbe6-ba1d-405f-962e-2f55d60a33f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:21.168112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e302bbe6-ba1d-405f-962e-2f55d60a33f9 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:21.184102 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c852d-1042-4ce1-9944-c4f07962abd2 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:21.187644 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:21.193265 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:21.231080 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527bacbd-f38a-477b-9b17-de8c1ffb0862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:21.232202 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bacbd-f38a-477b-9b17-de8c1ffb0862 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:21.232485 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bacbd-f38a-477b-9b17-de8c1ffb0862 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:21.232762 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bacbd-f38a-477b-9b17-de8c1ffb0862 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:21.245900 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527bacbd-f38a-477b-9b17-de8c1ffb0862 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:21.254825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-527bacbd-f38a-477b-9b17-de8c1ffb0862 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:25:21.254825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:21.254825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:21.254825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:21.254825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:21.254825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:21.254825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:21.254825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:21.254825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:21.254825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:21.254825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:21.254825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:21.254825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:21.254825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:21.254825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:21.254825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:21.256363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:21.256363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:21.256363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:21.256363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:21.256363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:21.256363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:21.256363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:21.256363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:21.256363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:21.256363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:21.256363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:21.256363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:21.256363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:21.256363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:21.256363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:21.256363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:21.258194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:21.258194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:21.258194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:21.258194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:21.258194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:21.258194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:21.258194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:21.258194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:21.258194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:21.258194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:21.258194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:25:21.258194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:25:21.258194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 8/14] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:25:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:25:21.262668 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527bacbd-f38a-477b-9b17-de8c1ffb0862 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:21.278064 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 7/15] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:25:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 04:25:21.283783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302bbe6-ba1d-405f-962e-2f55d60a33f9 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:21.372494 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527bacbd-f38a-477b-9b17-de8c1ffb0862 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:21.389288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302bbe6-ba1d-405f-962e-2f55d60a33f9 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:21.414177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b574611b-f1a1-4a39-878f-ca6de2ddf4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:21.415149 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574611b-f1a1-4a39-878f-ca6de2ddf4de None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:21.415602 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574611b-f1a1-4a39-878f-ca6de2ddf4de None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:21.416053 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574611b-f1a1-4a39-878f-ca6de2ddf4de None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:21.417099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b574611b-f1a1-4a39-878f-ca6de2ddf4de None admin] RBAC: Adding query filter params (name=admin) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:21.417616 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b574611b-f1a1-4a39-878f-ca6de2ddf4de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:21.420071 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b574611b-f1a1-4a39-878f-ca6de2ddf4de None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:21.420749 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:21.439253 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32154af-08a4-4c83-9b5c-69681cd2d090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:21.439903 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32154af-08a4-4c83-9b5c-69681cd2d090 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:21.440185 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32154af-08a4-4c83-9b5c-69681cd2d090 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:21.440464 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32154af-08a4-4c83-9b5c-69681cd2d090 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:21.440612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 9/16] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:25:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:25:21.449124 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b574611b-f1a1-4a39-878f-ca6de2ddf4de None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:21.453739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32154af-08a4-4c83-9b5c-69681cd2d090 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:21.455219 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32154af-08a4-4c83-9b5c-69681cd2d090 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:21.475084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-f32154af-08a4-4c83-9b5c-69681cd2d090 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:25:21.475084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:21.475084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:21.475084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:21.475084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:21.475084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:21.475084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:21.475084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:21.475084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:21.475084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:21.475084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:21.475084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:21.475084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:25:21.475084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:25:21.475084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:25:21.476522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:25:21.476522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:21.476522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:21.476522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:21.476522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:21.476522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:21.476522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:21.476522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:21.476522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:21.476522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:21.476522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:21.476522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:21.476522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:21.476522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:21.476522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:21.476522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:21.476522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:21.477505 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:25:21.477505 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:25:21.477505 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:25:21.477505 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:25:21.477505 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 8/17] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:25:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:25:21.479608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:21.483051 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32154af-08a4-4c83-9b5c-69681cd2d090 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:21.564377 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b574611b-f1a1-4a39-878f-ca6de2ddf4de None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:21.602006 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22b83f-93a0-4aa4-8f3c-d39339ad8d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:21.602666 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22b83f-93a0-4aa4-8f3c-d39339ad8d9a None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:21.602940 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22b83f-93a0-4aa4-8f3c-d39339ad8d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:21.603206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22b83f-93a0-4aa4-8f3c-d39339ad8d9a None admin] PATH_INFO: `/v3/users/fa79725fac2c4a528df2b060ede4922c` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:21.604830 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22b83f-93a0-4aa4-8f3c-d39339ad8d9a None admin] RBAC: Authorizing `identity:get_user(user_id=fa79725fac2c4a528df2b060ede4922c)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:21.606347 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22b83f-93a0-4aa4-8f3c-d39339ad8d9a None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:21.608018 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 10/18] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:21 2022] GET /identity/v3/users/fa79725fac2c4a528df2b060ede4922c => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:25:21.613774 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22b83f-93a0-4aa4-8f3c-d39339ad8d9a None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:21.707594 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32154af-08a4-4c83-9b5c-69681cd2d090 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:21.717180 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:21.724681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22b83f-93a0-4aa4-8f3c-d39339ad8d9a None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:21.746967 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56942dbf-b20a-4bbd-96a5-7a96aa248431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:21.747586 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56942dbf-b20a-4bbd-96a5-7a96aa248431 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:21.747867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56942dbf-b20a-4bbd-96a5-7a96aa248431 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:21.748150 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56942dbf-b20a-4bbd-96a5-7a96aa248431 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:21.748849 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56942dbf-b20a-4bbd-96a5-7a96aa248431 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:21.749158 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56942dbf-b20a-4bbd-96a5-7a96aa248431 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:21.750578 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56942dbf-b20a-4bbd-96a5-7a96aa248431 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:21.760782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab7f08-66a9-491d-b4f1-9a5f5a15672d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:21.761433 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab7f08-66a9-491d-b4f1-9a5f5a15672d None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:21.761747 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab7f08-66a9-491d-b4f1-9a5f5a15672d None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:21.762013 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab7f08-66a9-491d-b4f1-9a5f5a15672d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:21.763181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab7f08-66a9-491d-b4f1-9a5f5a15672d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:21.764624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab7f08-66a9-491d-b4f1-9a5f5a15672d None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:21.766228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 11/19] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:25:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:25:21.771580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 9/20] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:25:21.772212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab7f08-66a9-491d-b4f1-9a5f5a15672d None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:21.884719 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab7f08-66a9-491d-b4f1-9a5f5a15672d None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:21.924624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a1e1a-08a2-4133-b035-6353d31271a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:21.925250 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a1e1a-08a2-4133-b035-6353d31271a7 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:21.925526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a1e1a-08a2-4133-b035-6353d31271a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:21.925828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a1e1a-08a2-4133-b035-6353d31271a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:21.926626 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a1e1a-08a2-4133-b035-6353d31271a7 None admin] RBAC: Adding query filter params (role.id=4a6517e5890a49718303007dac0756c7, scope.domain.id=default, user.id=fa79725fac2c4a528df2b060ede4922c) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:21.926928 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a1e1a-08a2-4133-b035-6353d31271a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:21.928373 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a1e1a-08a2-4133-b035-6353d31271a7 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:21.938011 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 12/21] 10.222.0.47 () {56 vars in 1322 bytes} [Thu Aug 4 04:25:21 2022] GET /identity/v3/role_assignments?user.id=fa79725fac2c4a528df2b060ede4922c&scope.domain.id=default&role.id=4a6517e5890a49718303007dac0756c7 => generated 233 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:25:24.435437 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ba510-77e6-4646-9c3d-0db397a20cac None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:24.436479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ba510-77e6-4646-9c3d-0db397a20cac None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:24.436986 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ba510-77e6-4646-9c3d-0db397a20cac None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:24.438860 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 10/22] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:24.443954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07607f-3443-4132-a2b3-e894b55bc2e5 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:24.445899 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07607f-3443-4132-a2b3-e894b55bc2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:24.446384 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07607f-3443-4132-a2b3-e894b55bc2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:24.471314 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-0c07607f-3443-4132-a2b3-e894b55bc2e5 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:24.472868 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-0c07607f-3443-4132-a2b3-e894b55bc2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:24.501148 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:24.579471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 13/23] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:24.612106 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ba510-77e6-4646-9c3d-0db397a20cac None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:24.662077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:24.748889 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ba510-77e6-4646-9c3d-0db397a20cac None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:24.768349 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110bb01-2792-4359-8832-fc3b767898d1 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:24.768858 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110bb01-2792-4359-8832-fc3b767898d1 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:24.769271 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110bb01-2792-4359-8832-fc3b767898d1 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:24.770880 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 14/24] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:25:24.777923 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e1826-8fcd-4f5e-af35-d72445b514cf None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:24.778395 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e1826-8fcd-4f5e-af35-d72445b514cf None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:24.778835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e1826-8fcd-4f5e-af35-d72445b514cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:24.803111 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a4092-3a10-41bd-b049-8746dc101ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:24.805578 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a4092-3a10-41bd-b049-8746dc101ffc None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:24.805578 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a4092-3a10-41bd-b049-8746dc101ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:24.805578 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a4092-3a10-41bd-b049-8746dc101ffc None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:24.805578 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31a4092-3a10-41bd-b049-8746dc101ffc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:24.807009 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31a4092-3a10-41bd-b049-8746dc101ffc None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:24.809916 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-867e1826-8fcd-4f5e-af35-d72445b514cf None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:24.811258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-867e1826-8fcd-4f5e-af35-d72445b514cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:24.951910 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 15/25] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 04 04:25:24.969193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f1aa7-1117-405c-8f26-75dd66c65d2e None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:24.969725 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f1aa7-1117-405c-8f26-75dd66c65d2e None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:24.970185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f1aa7-1117-405c-8f26-75dd66c65d2e None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:24.971860 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 16/26] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:25:24.979556 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e8ac1-2631-4541-a5e1-cf61c039a480 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:24.980207 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e8ac1-2631-4541-a5e1-cf61c039a480 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:24.980625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e8ac1-2631-4541-a5e1-cf61c039a480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:24.988050 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 11/27] 10.222.0.47 () {60 vars in 1131 bytes} [Thu Aug 4 04:25:24 2022] POST /identity/v3/roles => generated 219 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:25:25.000241 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a4092-3a10-41bd-b049-8746dc101ffc None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:25.038077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-8f7e8ac1-2631-4541-a5e1-cf61c039a480 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:25.039713 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-8f7e8ac1-2631-4541-a5e1-cf61c039a480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:25.200907 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 17/28] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:25:25.248707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:25.260323 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a4092-3a10-41bd-b049-8746dc101ffc None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:25.277969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e8ac1-2631-4541-a5e1-cf61c039a480 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:25.326458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:25.347046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba90a3e-7624-49ac-9931-d49c1abcb1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:25.347706 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba90a3e-7624-49ac-9931-d49c1abcb1d7 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:25.347980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba90a3e-7624-49ac-9931-d49c1abcb1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:25.348274 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba90a3e-7624-49ac-9931-d49c1abcb1d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:25.376545 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba90a3e-7624-49ac-9931-d49c1abcb1d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:25.392209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-9ba90a3e-7624-49ac-9931-d49c1abcb1d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:25:25.392209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:25.392209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:25.392209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:25.392209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:25.392209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:25.392209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:25.392209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:25.392209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:25.392209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:25.392209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:25.392209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:25.392209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:25.392209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:25.392209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:25.392209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:25.393893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:25.393893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:25.393893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:25.393893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:25.393893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:25.393893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:25.393893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:25.393893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:25.393893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:25.393893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:25.393893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:25.393893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:25.393893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:25.393893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:25.393893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:25.393893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:25.395365 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:25.395365 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:25.395365 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:25.395365 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:25.395365 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:25.395365 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:25.395365 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:25.395365 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:25.395365 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:25.395365 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:25.395365 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:25:25.395365 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:25:25.395365 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 12/29] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:25:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:25:25.403391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba90a3e-7624-49ac-9931-d49c1abcb1d7 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:25.549893 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e8ac1-2631-4541-a5e1-cf61c039a480 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:25.655684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba90a3e-7624-49ac-9931-d49c1abcb1d7 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:25.716006 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4022d0-99a8-4a2e-9c0a-633cb0336470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:25.716658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4022d0-99a8-4a2e-9c0a-633cb0336470 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:25.716945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4022d0-99a8-4a2e-9c0a-633cb0336470 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:25.717224 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4022d0-99a8-4a2e-9c0a-633cb0336470 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:25.718048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4022d0-99a8-4a2e-9c0a-633cb0336470 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:25.718372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4022d0-99a8-4a2e-9c0a-633cb0336470 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:25.719957 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4022d0-99a8-4a2e-9c0a-633cb0336470 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:25.734128 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 13/30] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:25:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:25:25.735339 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cfdf0-74c5-4d9d-bd91-6b5ff323d720 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:25.735630 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cfdf0-74c5-4d9d-bd91-6b5ff323d720 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:25.735908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cfdf0-74c5-4d9d-bd91-6b5ff323d720 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:25.737417 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 14/31] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:25.742544 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671cfdf0-74c5-4d9d-bd91-6b5ff323d720 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:25.764340 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028b021-9c39-44e4-b5d8-72b08b192675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:25.765225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028b021-9c39-44e4-b5d8-72b08b192675 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:25.765682 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028b021-9c39-44e4-b5d8-72b08b192675 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:25.766094 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028b021-9c39-44e4-b5d8-72b08b192675 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:25.773864 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:25.789789 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c028b021-9c39-44e4-b5d8-72b08b192675 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:25.791537 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c028b021-9c39-44e4-b5d8-72b08b192675 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:25.809884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-c028b021-9c39-44e4-b5d8-72b08b192675 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:25:25.809884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:25.809884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:25.809884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:25.809884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:25.809884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:25.809884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:25.809884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:25.809884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:25.809884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:25.809884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:25.809884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:25.809884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:25:25.809884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:25:25.809884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:25:25.810883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:25:25.810883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:25.810883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:25.810883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:25.810883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:25.810883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:25.810883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:25.810883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:25.810883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:25.810883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:25.810883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:25.810883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:25.810883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:25.810883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:25.810883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:25.810883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:25.810883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:25.811744 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:25:25.811744 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:25:25.811744 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:25:25.811744 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:25:25.813263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 18/32] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:25:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 536 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:25:25.815299 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5395aa3-6902-491d-a50d-cbc4b874d177 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:25.815732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5395aa3-6902-491d-a50d-cbc4b874d177 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:25.816145 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5395aa3-6902-491d-a50d-cbc4b874d177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:25.836805 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-d5395aa3-6902-491d-a50d-cbc4b874d177 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:25.838046 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d5395aa3-6902-491d-a50d-cbc4b874d177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:25.913846 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:25.914307 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671cfdf0-74c5-4d9d-bd91-6b5ff323d720 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:25.920640 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 19/33] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:25.921703 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5395aa3-6902-491d-a50d-cbc4b874d177 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:25.966838 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d8579-5bf0-4867-8d71-90c0e1aefa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:25.967840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d8579-5bf0-4867-8d71-90c0e1aefa99 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:25.968299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d8579-5bf0-4867-8d71-90c0e1aefa99 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:25.968758 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d8579-5bf0-4867-8d71-90c0e1aefa99 None admin] PATH_INFO: `/v3/users/fa79725fac2c4a528df2b060ede4922c` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:25.970688 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4d8579-5bf0-4867-8d71-90c0e1aefa99 None admin] RBAC: Authorizing `identity:get_user(user_id=fa79725fac2c4a528df2b060ede4922c)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:25.973231 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4d8579-5bf0-4867-8d71-90c0e1aefa99 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:25.976694 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 15/34] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:25 2022] GET /identity/v3/users/fa79725fac2c4a528df2b060ede4922c => generated 247 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:25.978419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2cdd2-8a6d-4e35-93c7-448e6c373baf None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:25.978885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2cdd2-8a6d-4e35-93c7-448e6c373baf None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:25.979345 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2cdd2-8a6d-4e35-93c7-448e6c373baf None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:25.981496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 16/35] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:25.986467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2cdd2-8a6d-4e35-93c7-448e6c373baf None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:26.078743 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5395aa3-6902-491d-a50d-cbc4b874d177 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:26.112902 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:26.129188 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f08887-153c-4e57-a16f-3dfa595aaf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:26.129842 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f08887-153c-4e57-a16f-3dfa595aaf62 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:26.130443 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f08887-153c-4e57-a16f-3dfa595aaf62 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:26.130720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f08887-153c-4e57-a16f-3dfa595aaf62 None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:26.132313 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f08887-153c-4e57-a16f-3dfa595aaf62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:26.133078 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f08887-153c-4e57-a16f-3dfa595aaf62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:26.134481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f08887-153c-4e57-a16f-3dfa595aaf62 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:26.134827 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2cdd2-8a6d-4e35-93c7-448e6c373baf None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:26.165587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 20/36] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:25:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:26.165587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f08887-153c-4e57-a16f-3dfa595aaf62 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:26.173282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40f555-2194-462e-99ec-a040e0eaf79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:26.173926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40f555-2194-462e-99ec-a040e0eaf79b None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:26.174210 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40f555-2194-462e-99ec-a040e0eaf79b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:26.174486 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40f555-2194-462e-99ec-a040e0eaf79b None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:26.175206 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40f555-2194-462e-99ec-a040e0eaf79b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:26.176595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40f555-2194-462e-99ec-a040e0eaf79b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:26.194165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 17/37] 10.222.0.47 () {60 vars in 1131 bytes} [Thu Aug 4 04:25:25 2022] POST /identity/v3/roles => generated 223 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:25:26.195417 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4d8ea-22d8-4604-b747-6fb596a02048 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:26.195693 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4d8ea-22d8-4604-b747-6fb596a02048 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:26.195970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4d8ea-22d8-4604-b747-6fb596a02048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:26.225636 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-f8b4d8ea-22d8-4604-b747-6fb596a02048 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:26.227332 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f8b4d8ea-22d8-4604-b747-6fb596a02048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:26.277128 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:26.327898 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 18/38] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:26.327898 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4d8ea-22d8-4604-b747-6fb596a02048 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:26.350086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f08887-153c-4e57-a16f-3dfa595aaf62 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:26.393016 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e49c2ca-5514-424c-9352-9489ca3048a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:26.393937 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49c2ca-5514-424c-9352-9489ca3048a8 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:26.394214 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49c2ca-5514-424c-9352-9489ca3048a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:26.394485 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49c2ca-5514-424c-9352-9489ca3048a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:26.395656 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e49c2ca-5514-424c-9352-9489ca3048a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:26.397348 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e49c2ca-5514-424c-9352-9489ca3048a8 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:26.399215 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 21/39] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:25:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:26.402114 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d19cf-fde9-4d67-882c-dba4d35bfd45 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:26.402114 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d19cf-fde9-4d67-882c-dba4d35bfd45 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:26.402114 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d19cf-fde9-4d67-882c-dba4d35bfd45 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:26.402114 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 22/40] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:26.407269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d19cf-fde9-4d67-882c-dba4d35bfd45 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:26.436015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:26.469611 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4d8ea-22d8-4604-b747-6fb596a02048 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:26.508920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2060ef-33a6-435e-a0c9-e5dbc3598fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:26.509576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2060ef-33a6-435e-a0c9-e5dbc3598fd6 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:26.510299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2060ef-33a6-435e-a0c9-e5dbc3598fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:26.510586 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2060ef-33a6-435e-a0c9-e5dbc3598fd6 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:26.511305 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2060ef-33a6-435e-a0c9-e5dbc3598fd6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:26.513155 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2060ef-33a6-435e-a0c9-e5dbc3598fd6 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:26.539381 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d19cf-fde9-4d67-882c-dba4d35bfd45 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:26.546448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 19/41] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Aug 4 04:25:26 2022] POST /identity/v3/projects => generated 301 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:25:26.547980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2060ef-33a6-435e-a0c9-e5dbc3598fd6 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:26.586352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:26.602736 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7da80-1363-4f1c-9727-990dba243520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:26.603667 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7da80-1363-4f1c-9727-990dba243520 None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:26.604166 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7da80-1363-4f1c-9727-990dba243520 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:26.604675 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7da80-1363-4f1c-9727-990dba243520 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:26.606098 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7da80-1363-4f1c-9727-990dba243520 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:26.608649 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7da80-1363-4f1c-9727-990dba243520 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:26.726180 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2060ef-33a6-435e-a0c9-e5dbc3598fd6 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:26.755334 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:26.767174 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5e4ca-4b0d-4715-a731-e74613876047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:26.769189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5e4ca-4b0d-4715-a731-e74613876047 None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:26.769473 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5e4ca-4b0d-4715-a731-e74613876047 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:26.769799 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5e4ca-4b0d-4715-a731-e74613876047 None admin] PATH_INFO: `/v3/domains/default/users/fa79725fac2c4a528df2b060ede4922c/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:26.771913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b5e4ca-4b0d-4715-a731-e74613876047 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=fa79725fac2c4a528df2b060ede4922c, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:26.774228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b5e4ca-4b0d-4715-a731-e74613876047 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:26.790221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 20/42] 10.222.0.47 () {58 vars in 1361 bytes} [Thu Aug 4 04:25:26 2022] PUT /identity/v3/domains/default/users/fa79725fac2c4a528df2b060ede4922c/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:25:26.791414 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee68f83-2552-4b93-8e5f-12ea3b008067 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:26.791708 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee68f83-2552-4b93-8e5f-12ea3b008067 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:26.791986 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee68f83-2552-4b93-8e5f-12ea3b008067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:26.799457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 23/43] 10.222.0.47 () {60 vars in 1131 bytes} [Thu Aug 4 04:25:26 2022] POST /identity/v3/roles => generated 225 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:25:26.812810 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-1ee68f83-2552-4b93-8e5f-12ea3b008067 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:26.814381 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1ee68f83-2552-4b93-8e5f-12ea3b008067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:26.897664 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 21/44] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:26.921024 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7da80-1363-4f1c-9727-990dba243520 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:27.039619 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7da80-1363-4f1c-9727-990dba243520 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:27.076330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a6513-423f-4d89-ad8f-80af25655afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:27.076979 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a6513-423f-4d89-ad8f-80af25655afc None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:27.077260 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a6513-423f-4d89-ad8f-80af25655afc None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:27.077535 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a6513-423f-4d89-ad8f-80af25655afc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:27.078809 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a6513-423f-4d89-ad8f-80af25655afc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:27.080358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a6513-423f-4d89-ad8f-80af25655afc None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:27.081998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 24/45] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:25:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:27.087826 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee68f83-2552-4b93-8e5f-12ea3b008067 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:27.189098 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee68f83-2552-4b93-8e5f-12ea3b008067 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:27.206207 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:27.224712 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf007c-a9ab-4102-aaab-86ee8e05afee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:27.225933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf007c-a9ab-4102-aaab-86ee8e05afee None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:27.226209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf007c-a9ab-4102-aaab-86ee8e05afee None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:27.226550 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf007c-a9ab-4102-aaab-86ee8e05afee None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:27.227296 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcf007c-a9ab-4102-aaab-86ee8e05afee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:27.228778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcf007c-a9ab-4102-aaab-86ee8e05afee None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:27.256963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 22/46] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Aug 4 04:25:27 2022] POST /identity/v3/projects => generated 312 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:25:28.853969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120223e-3c87-44e4-85ad-04d99b6f4b27 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:28.854683 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120223e-3c87-44e4-85ad-04d99b6f4b27 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:28.854683 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120223e-3c87-44e4-85ad-04d99b6f4b27 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:28.855883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 25/47] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:28.861840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442744d-9544-4001-a6f3-6c764ba77b22 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:28.862311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442744d-9544-4001-a6f3-6c764ba77b22 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:28.862467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442744d-9544-4001-a6f3-6c764ba77b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:28.882189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-c442744d-9544-4001-a6f3-6c764ba77b22 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:28.883399 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c442744d-9544-4001-a6f3-6c764ba77b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:28.955208 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 23/48] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:28.979089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8120223e-3c87-44e4-85ad-04d99b6f4b27 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:29.039376 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:29.087401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8120223e-3c87-44e4-85ad-04d99b6f4b27 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:29.125717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3268909-ba4e-4872-8b7c-43bb6a982d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:29.126175 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3268909-ba4e-4872-8b7c-43bb6a982d7f None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:29.126427 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3268909-ba4e-4872-8b7c-43bb6a982d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:29.126702 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3268909-ba4e-4872-8b7c-43bb6a982d7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:29.140528 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3268909-ba4e-4872-8b7c-43bb6a982d7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:29.148414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-c3268909-ba4e-4872-8b7c-43bb6a982d7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:25:29.148414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:29.148414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:29.148414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:29.148414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:29.148414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:29.148414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:29.148414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:29.148414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:29.148414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:29.148414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:29.148414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:29.148414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:29.148414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:29.148414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:29.148414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:29.149480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:29.149480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:29.149480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:29.149480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:29.149480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:29.149480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:29.149480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:29.149480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:29.149480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:29.149480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:29.149480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:29.149480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:29.149480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:29.149480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:29.149480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:29.149480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:29.151030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:29.151030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:29.151030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:29.151030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:29.151030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:29.151030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:29.151030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:29.151030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:29.151030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:29.151030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:29.151030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:25:29.151030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:25:29.151030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 26/49] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:25:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:25:29.156769 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442744d-9544-4001-a6f3-6c764ba77b22 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:29.267680 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:29.272368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442744d-9544-4001-a6f3-6c764ba77b22 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:29.309608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de5f38-c718-4412-a72a-95b39fb0e146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:29.310247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de5f38-c718-4412-a72a-95b39fb0e146 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:29.310624 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de5f38-c718-4412-a72a-95b39fb0e146 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:29.310802 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de5f38-c718-4412-a72a-95b39fb0e146 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:29.311536 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de5f38-c718-4412-a72a-95b39fb0e146 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:29.311844 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de5f38-c718-4412-a72a-95b39fb0e146 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:29.313252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de5f38-c718-4412-a72a-95b39fb0e146 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:29.322378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 24/50] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:25:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:29.328626 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3268909-ba4e-4872-8b7c-43bb6a982d7f None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:29.365889 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b07b4a-936f-4220-99e5-c7a72b59f9d3 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:29.366321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b07b4a-936f-4220-99e5-c7a72b59f9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:29.366617 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b07b4a-936f-4220-99e5-c7a72b59f9d3 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:29.367978 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 25/51] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:25:29.373253 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb56ac-f904-4559-8149-7281143fdfef None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:29.373567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb56ac-f904-4559-8149-7281143fdfef None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:29.373842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb56ac-f904-4559-8149-7281143fdfef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:29.394941 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-2feb56ac-f904-4559-8149-7281143fdfef None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:29.396137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2feb56ac-f904-4559-8149-7281143fdfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:29.435212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3268909-ba4e-4872-8b7c-43bb6a982d7f None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:29.451250 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:29.473463 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643284c-9ed1-42d3-a684-2325a7933e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:29.474199 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643284c-9ed1-42d3-a684-2325a7933e12 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:29.474410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 26/52] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:25:29.474740 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643284c-9ed1-42d3-a684-2325a7933e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:29.475055 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643284c-9ed1-42d3-a684-2325a7933e12 None admin] PATH_INFO: `/v3/users/fa79725fac2c4a528df2b060ede4922c` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:29.478402 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643284c-9ed1-42d3-a684-2325a7933e12 None admin] RBAC: Authorizing `identity:get_user(user_id=fa79725fac2c4a528df2b060ede4922c)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:29.479949 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643284c-9ed1-42d3-a684-2325a7933e12 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:29.481642 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 27/53] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:29 2022] GET /identity/v3/users/fa79725fac2c4a528df2b060ede4922c => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:25:29.487145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb56ac-f904-4559-8149-7281143fdfef None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:29.497953 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643284c-9ed1-42d3-a684-2325a7933e12 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:29.594541 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:29.605957 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb56ac-f904-4559-8149-7281143fdfef None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:29.607023 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643284c-9ed1-42d3-a684-2325a7933e12 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:29.642540 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f6f68-2e99-4ae6-beb8-b01ed7dd4adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:29.643192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f6f68-2e99-4ae6-beb8-b01ed7dd4adf None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:29.643478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f6f68-2e99-4ae6-beb8-b01ed7dd4adf None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:29.643754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f6f68-2e99-4ae6-beb8-b01ed7dd4adf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:29.644292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505faba9-ab82-4547-8901-9a5551f47a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:29.644934 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505faba9-ab82-4547-8901-9a5551f47a54 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:29.645250 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f6f68-2e99-4ae6-beb8-b01ed7dd4adf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:29.645357 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505faba9-ab82-4547-8901-9a5551f47a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:29.645481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505faba9-ab82-4547-8901-9a5551f47a54 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:29.646611 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f6f68-2e99-4ae6-beb8-b01ed7dd4adf None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:29.646721 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505faba9-ab82-4547-8901-9a5551f47a54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:29.648837 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505faba9-ab82-4547-8901-9a5551f47a54 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:29.649244 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 27/54] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:25:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:25:29.657536 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 28/55] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:25:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:25:29.657782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f6f68-2e99-4ae6-beb8-b01ed7dd4adf None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:29.658411 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505faba9-ab82-4547-8901-9a5551f47a54 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:29.709314 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:29.764906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f6f68-2e99-4ae6-beb8-b01ed7dd4adf None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:29.769288 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505faba9-ab82-4547-8901-9a5551f47a54 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:29.818653 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d919ed-1000-4d45-88f3-6f841c38a69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:29.819686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d919ed-1000-4d45-88f3-6f841c38a69f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:29.820269 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d919ed-1000-4d45-88f3-6f841c38a69f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:29.821015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d919ed-1000-4d45-88f3-6f841c38a69f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:29.821806 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81269cb-1e79-487c-b38a-e7fe3a85f2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:29.822601 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81269cb-1e79-487c-b38a-e7fe3a85f2ca None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:29.822756 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d919ed-1000-4d45-88f3-6f841c38a69f None admin] RBAC: Adding query filter params (role.id=4a6517e5890a49718303007dac0756c7, scope.domain.id=default, user.id=fa79725fac2c4a528df2b060ede4922c) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:29.822997 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81269cb-1e79-487c-b38a-e7fe3a85f2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:29.823370 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d919ed-1000-4d45-88f3-6f841c38a69f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:29.823524 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81269cb-1e79-487c-b38a-e7fe3a85f2ca None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:29.824408 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81269cb-1e79-487c-b38a-e7fe3a85f2ca None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:29.826452 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d919ed-1000-4d45-88f3-6f841c38a69f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:29.826886 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81269cb-1e79-487c-b38a-e7fe3a85f2ca None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:29.839568 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 28/56] 10.222.0.47 () {56 vars in 1322 bytes} [Thu Aug 4 04:25:29 2022] GET /identity/v3/role_assignments?user.id=fa79725fac2c4a528df2b060ede4922c&scope.domain.id=default&role.id=4a6517e5890a49718303007dac0756c7 => generated 530 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:25:29.869851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 29/57] 10.222.0.47 () {60 vars in 1140 bytes} [Thu Aug 4 04:25:29 2022] POST /identity/v3/projects => generated 298 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 04:25:31.997207 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d66f1-eef0-49ad-ad1e-10436b898d34 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:31.997826 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d66f1-eef0-49ad-ad1e-10436b898d34 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:31.997826 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d66f1-eef0-49ad-ad1e-10436b898d34 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:31.999165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 29/58] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:32.004443 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fe3f2-d7ce-4bce-8114-4ce01a516aa7 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:32.004878 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fe3f2-d7ce-4bce-8114-4ce01a516aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:32.004999 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fe3f2-d7ce-4bce-8114-4ce01a516aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:32.024140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-617fe3f2-d7ce-4bce-8114-4ce01a516aa7 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:32.025690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-617fe3f2-d7ce-4bce-8114-4ce01a516aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:32.095940 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 30/59] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:25:32.121723 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d66f1-eef0-49ad-ad1e-10436b898d34 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:32.148001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:32.229628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d66f1-eef0-49ad-ad1e-10436b898d34 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:32.263799 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db216f3b-c0d2-41a1-b427-7f931c85e1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:32.264352 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db216f3b-c0d2-41a1-b427-7f931c85e1ff None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:32.264630 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db216f3b-c0d2-41a1-b427-7f931c85e1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:32.264905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db216f3b-c0d2-41a1-b427-7f931c85e1ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:32.266098 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db216f3b-c0d2-41a1-b427-7f931c85e1ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:32.267523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db216f3b-c0d2-41a1-b427-7f931c85e1ff None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:32.269091 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 30/60] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:25:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:32.274962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fe3f2-d7ce-4bce-8114-4ce01a516aa7 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:32.307170 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:32.398764 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fe3f2-d7ce-4bce-8114-4ce01a516aa7 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:32.435188 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9e3c3-6af0-4f31-b792-854829229c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:32.435799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9e3c3-6af0-4f31-b792-854829229c44 None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:32.436078 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9e3c3-6af0-4f31-b792-854829229c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:32.436343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9e3c3-6af0-4f31-b792-854829229c44 None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:32.437038 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c9e3c3-6af0-4f31-b792-854829229c44 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:32.438887 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c9e3c3-6af0-4f31-b792-854829229c44 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:32.470323 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 31/61] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Aug 4 04:25:32 2022] POST /identity/v3/users => generated 275 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:25:35.935265 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f331c68-1609-452b-91b1-52811ac3c0dd None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:35.936188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f331c68-1609-452b-91b1-52811ac3c0dd None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:35.936618 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f331c68-1609-452b-91b1-52811ac3c0dd None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:35.938145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 31/62] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:35.943537 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b63675-bdc2-436a-acb3-9ca37a72ee54 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:35.944523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b63675-bdc2-436a-acb3-9ca37a72ee54 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:35.944965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b63675-bdc2-436a-acb3-9ca37a72ee54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:35.974514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-52b63675-bdc2-436a-acb3-9ca37a72ee54 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:35.976823 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-52b63675-bdc2-436a-acb3-9ca37a72ee54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:36.011629 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b21c8b-3939-43f8-bb7c-0121f64f7787 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:36.012218 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b21c8b-3939-43f8-bb7c-0121f64f7787 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:36.012694 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b21c8b-3939-43f8-bb7c-0121f64f7787 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:36.014303 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 32/63] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:36.019205 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad1c73-3b55-43c2-91c5-d2b8d9225484 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:36.019733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad1c73-3b55-43c2-91c5-d2b8d9225484 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:36.020219 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad1c73-3b55-43c2-91c5-d2b8d9225484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:36.058454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 32/64] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:36.059940 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-8dad1c73-3b55-43c2-91c5-d2b8d9225484 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:36.061166 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8dad1c73-3b55-43c2-91c5-d2b8d9225484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:36.062066 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41ccc7-1ea2-447f-ad8e-a865eade5003 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:36.062560 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41ccc7-1ea2-447f-ad8e-a865eade5003 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:36.063036 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41ccc7-1ea2-447f-ad8e-a865eade5003 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:36.064816 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 33/65] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:36.070681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ade1c-4a81-4efd-989e-0a15738ea45f None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:36.070970 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ade1c-4a81-4efd-989e-0a15738ea45f None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:36.071252 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ade1c-4a81-4efd-989e-0a15738ea45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:36.094339 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-906ade1c-4a81-4efd-989e-0a15738ea45f None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:36.095547 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-906ade1c-4a81-4efd-989e-0a15738ea45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:36.111101 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:36.143722 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 33/66] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:25:36.144373 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad1c73-3b55-43c2-91c5-d2b8d9225484 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:36.170929 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 34/67] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:25:36.182914 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ade1c-4a81-4efd-989e-0a15738ea45f None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:36.203901 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:36.290927 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad1c73-3b55-43c2-91c5-d2b8d9225484 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:36.297854 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ade1c-4a81-4efd-989e-0a15738ea45f None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:36.333619 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5c841-f1eb-46ba-967f-f0ee84ee4120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:36.335035 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5c841-f1eb-46ba-967f-f0ee84ee4120 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:36.335332 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5c841-f1eb-46ba-967f-f0ee84ee4120 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:36.335612 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5c841-f1eb-46ba-967f-f0ee84ee4120 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:36.337587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e3184-5b2a-4607-a38b-a92e0a7d3b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:36.338240 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e3184-5b2a-4607-a38b-a92e0a7d3b72 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:36.338521 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e3184-5b2a-4607-a38b-a92e0a7d3b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:36.338793 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e3184-5b2a-4607-a38b-a92e0a7d3b72 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:36.346009 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e5c841-f1eb-46ba-967f-f0ee84ee4120 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:36.348911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e3184-5b2a-4607-a38b-a92e0a7d3b72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:36.351264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-63e5c841-f1eb-46ba-967f-f0ee84ee4120 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:25:36.351264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:36.351264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:36.351264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:36.351264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:36.351264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:36.351264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:36.351264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:36.351264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:36.351264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:36.351264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:36.351264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:36.351264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:36.351264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:36.351264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:36.351264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:36.352393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:36.352393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:36.352393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:36.352393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:36.352393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:36.352393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:36.352393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:36.352393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:36.352393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:36.352393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:36.352393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:36.352393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:36.352393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:36.352393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:36.352393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:36.352393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:36.353187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:36.353187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:36.353187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:36.353187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:36.353187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:36.353187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:36.353187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:36.353187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:36.353187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:36.353187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:36.353187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:25:36.353187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:25:36.353187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 34/68] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:25:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:25:36.354130 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5c841-f1eb-46ba-967f-f0ee84ee4120 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:36.355722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-705e3184-5b2a-4607-a38b-a92e0a7d3b72 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:25:36.355722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:36.355722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:36.355722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:36.355722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:36.355722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:36.355722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:36.355722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:36.355722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:36.355722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:36.355722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:36.355722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:36.355722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:36.355722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:36.355722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:36.355722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:36.356492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:36.356492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:36.356492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:36.356492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:36.356492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:36.356492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:36.356492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:36.356492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:36.356492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:36.356492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:36.356492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:36.356492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:36.356492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:36.356492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:36.356492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:36.356492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:36.357220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:36.357220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:36.357220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:36.357220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:36.357220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:36.357220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:36.357220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:36.357220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:36.357220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:36.357220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:36.357220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:25:36.357220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:25:36.357840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 35/69] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Aug 4 04:25:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:25:36.359772 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d930a8c-fb01-4708-bebc-b4374d694fb3 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:36.360067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d930a8c-fb01-4708-bebc-b4374d694fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:36.360347 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d930a8c-fb01-4708-bebc-b4374d694fb3 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:36.361708 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 36/70] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:36.367310 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588e10b-e8ba-4161-b9a1-f18e90e7cdbc None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:36.367602 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588e10b-e8ba-4161-b9a1-f18e90e7cdbc None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:36.367882 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588e10b-e8ba-4161-b9a1-f18e90e7cdbc None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:36.369256 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 37/71] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:36.369868 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588e10b-e8ba-4161-b9a1-f18e90e7cdbc None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:36.386571 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:36.439546 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:36.460294 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5c841-f1eb-46ba-967f-f0ee84ee4120 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:36.478630 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588e10b-e8ba-4161-b9a1-f18e90e7cdbc None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:36.493461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990e013-9c3c-4fd8-91e1-44b7e549e26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:36.494129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990e013-9c3c-4fd8-91e1-44b7e549e26b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:36.494407 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990e013-9c3c-4fd8-91e1-44b7e549e26b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:36.494677 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990e013-9c3c-4fd8-91e1-44b7e549e26b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:36.504880 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990e013-9c3c-4fd8-91e1-44b7e549e26b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:36.510447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-5990e013-9c3c-4fd8-91e1-44b7e549e26b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:25:36.510447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:36.510447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:36.510447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:36.510447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:36.510447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:36.510447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:36.510447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:36.510447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:36.510447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:36.510447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:36.510447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:36.510447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:36.510447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:36.510447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:36.510447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:36.511298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:36.511298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:36.511298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:36.511298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:36.511298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:36.511298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:36.511298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:36.511298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:36.511298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:36.511298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:36.511298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:36.511298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:36.511298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:36.511298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:36.511298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:36.511298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:36.512038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:36.512038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:36.512038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:36.512038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:36.512038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:36.512038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:36.512038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:36.512038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:36.512038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:36.512038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:36.512038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:25:36.512038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:25:36.512038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 35/72] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:25:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:25:36.512660 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990e013-9c3c-4fd8-91e1-44b7e549e26b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:36.520959 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709cc8ec-0b43-47db-a05d-35581db517eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:36.521752 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cc8ec-0b43-47db-a05d-35581db517eb None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:36.521958 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cc8ec-0b43-47db-a05d-35581db517eb None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:36.522242 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cc8ec-0b43-47db-a05d-35581db517eb None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:36.522972 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709cc8ec-0b43-47db-a05d-35581db517eb None admin] RBAC: Adding query filter params (name=member) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:36.523284 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709cc8ec-0b43-47db-a05d-35581db517eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:36.524675 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709cc8ec-0b43-47db-a05d-35581db517eb None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:36.533311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 38/73] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:25:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:36.534500 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864fc796-6bf0-4bfc-908a-fe8221e965ac None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:36.534789 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864fc796-6bf0-4bfc-908a-fe8221e965ac None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:36.535068 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864fc796-6bf0-4bfc-908a-fe8221e965ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:36.554822 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-864fc796-6bf0-4bfc-908a-fe8221e965ac None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:36.556263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-864fc796-6bf0-4bfc-908a-fe8221e965ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:36.623285 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990e013-9c3c-4fd8-91e1-44b7e549e26b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:36.623948 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:36.631792 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 39/74] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:36.633762 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465808e-f9eb-4a1f-9e77-a3fca57366f4 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:36.634083 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465808e-f9eb-4a1f-9e77-a3fca57366f4 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:36.634364 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465808e-f9eb-4a1f-9e77-a3fca57366f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:36.653039 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-e465808e-f9eb-4a1f-9e77-a3fca57366f4 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:36.654261 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-e465808e-f9eb-4a1f-9e77-a3fca57366f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:36.658885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d05585-1499-4d84-8304-149d50eda1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:36.659500 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d05585-1499-4d84-8304-149d50eda1fc None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:36.659781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d05585-1499-4d84-8304-149d50eda1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:36.660055 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d05585-1499-4d84-8304-149d50eda1fc None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:36.660775 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d05585-1499-4d84-8304-149d50eda1fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:36.661080 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d05585-1499-4d84-8304-149d50eda1fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:36.662705 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d05585-1499-4d84-8304-149d50eda1fc None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:36.670671 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 36/75] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:25:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:36.674339 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d05585-1499-4d84-8304-149d50eda1fc None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:36.677287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:36.721937 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 40/76] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:36.722763 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465808e-f9eb-4a1f-9e77-a3fca57366f4 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:36.777282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d05585-1499-4d84-8304-149d50eda1fc None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:36.812699 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8f146-7172-4d27-a62a-4c101a8d1814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:36.813408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8f146-7172-4d27-a62a-4c101a8d1814 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:36.813733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8f146-7172-4d27-a62a-4c101a8d1814 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:36.814007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8f146-7172-4d27-a62a-4c101a8d1814 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:36.814734 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8f146-7172-4d27-a62a-4c101a8d1814 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:36.815042 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8f146-7172-4d27-a62a-4c101a8d1814 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:36.816426 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8f146-7172-4d27-a62a-4c101a8d1814 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:36.825807 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 37/77] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:25:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:36.826492 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8f146-7172-4d27-a62a-4c101a8d1814 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:36.832169 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465808e-f9eb-4a1f-9e77-a3fca57366f4 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:36.860888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:36.868373 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcd98f-dd19-4c34-a6ca-69c329eaf861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:36.869025 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcd98f-dd19-4c34-a6ca-69c329eaf861 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:36.869309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcd98f-dd19-4c34-a6ca-69c329eaf861 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:36.869643 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcd98f-dd19-4c34-a6ca-69c329eaf861 None admin] PATH_INFO: `/v3/users/e82fe3500b4c40718ae00567a41063b9` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:36.884619 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fcd98f-dd19-4c34-a6ca-69c329eaf861 None admin] RBAC: Authorizing `identity:get_user(user_id=e82fe3500b4c40718ae00567a41063b9)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:36.886128 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fcd98f-dd19-4c34-a6ca-69c329eaf861 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:36.887783 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 41/78] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:36 2022] GET /identity/v3/users/e82fe3500b4c40718ae00567a41063b9 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:36.888733 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcd98f-dd19-4c34-a6ca-69c329eaf861 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:36.893568 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:36.932865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8f146-7172-4d27-a62a-4c101a8d1814 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:36.965837 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09bfdb-e072-4df0-84e4-c4bca9921327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:36.966466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09bfdb-e072-4df0-84e4-c4bca9921327 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:36.966748 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09bfdb-e072-4df0-84e4-c4bca9921327 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:36.967047 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09bfdb-e072-4df0-84e4-c4bca9921327 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:36.968228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09bfdb-e072-4df0-84e4-c4bca9921327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:36.969723 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09bfdb-e072-4df0-84e4-c4bca9921327 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:36.971308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 38/79] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:25:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:36.972128 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09bfdb-e072-4df0-84e4-c4bca9921327 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:36.989850 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcd98f-dd19-4c34-a6ca-69c329eaf861 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:37.024109 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cafff1a-7a39-4ddf-901c-d31d18c52dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:37.024746 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafff1a-7a39-4ddf-901c-d31d18c52dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:37.025028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafff1a-7a39-4ddf-901c-d31d18c52dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:37.025308 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafff1a-7a39-4ddf-901c-d31d18c52dd1 None admin] PATH_INFO: `/v3/users/e82fe3500b4c40718ae00567a41063b9` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:37.026507 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cafff1a-7a39-4ddf-901c-d31d18c52dd1 None admin] RBAC: Authorizing `identity:get_user(user_id=e82fe3500b4c40718ae00567a41063b9)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:37.027932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cafff1a-7a39-4ddf-901c-d31d18c52dd1 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:37.029705 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 42/80] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:36 2022] GET /identity/v3/users/e82fe3500b4c40718ae00567a41063b9 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:37.030296 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cafff1a-7a39-4ddf-901c-d31d18c52dd1 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:37.085726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09bfdb-e072-4df0-84e4-c4bca9921327 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:37.086815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:37.114889 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:37.120981 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487efb1f-906d-45d8-b281-cc1b4fb65172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:37.121650 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487efb1f-906d-45d8-b281-cc1b4fb65172 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:37.121935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487efb1f-906d-45d8-b281-cc1b4fb65172 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:37.122211 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487efb1f-906d-45d8-b281-cc1b4fb65172 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:37.133095 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487efb1f-906d-45d8-b281-cc1b4fb65172 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:37.133236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cafff1a-7a39-4ddf-901c-d31d18c52dd1 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:37.139082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-487efb1f-906d-45d8-b281-cc1b4fb65172 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:25:37.139082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:37.139082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:37.139082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:37.139082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:37.139082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:37.139082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:37.139082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:37.139082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:37.139082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:37.139082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:37.139082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:37.139082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:37.139082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:37.139082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:37.139082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:37.139913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:37.139913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:37.139913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:37.139913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:37.139913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:37.139913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:37.139913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:37.139913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:37.139913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:37.139913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:37.139913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:37.139913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:37.139913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:37.139913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:37.139913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:37.139913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:37.140654 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:37.140654 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:37.140654 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:37.140654 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:37.140654 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:37.140654 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:37.140654 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:37.140654 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:37.140654 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:37.140654 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:37.140654 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:25:37.140654 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:25:37.140654 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 39/81] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:25:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:25:37.141267 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487efb1f-906d-45d8-b281-cc1b4fb65172 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:37.169750 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c41ba-daba-412f-960e-e179d3dc979b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:37.170389 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c41ba-daba-412f-960e-e179d3dc979b None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:37.170667 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c41ba-daba-412f-960e-e179d3dc979b None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:37.171206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c41ba-daba-412f-960e-e179d3dc979b None admin] PATH_INFO: `/v3/users/fa79725fac2c4a528df2b060ede4922c` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:37.172407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c41ba-daba-412f-960e-e179d3dc979b None admin] RBAC: Authorizing `identity:get_user(user_id=fa79725fac2c4a528df2b060ede4922c)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:37.173912 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c41ba-daba-412f-960e-e179d3dc979b None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:37.175543 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 43/82] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:37 2022] GET /identity/v3/users/fa79725fac2c4a528df2b060ede4922c => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:37.176229 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c41ba-daba-412f-960e-e179d3dc979b None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:37.243683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487efb1f-906d-45d8-b281-cc1b4fb65172 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:37.276503 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c41ba-daba-412f-960e-e179d3dc979b None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:37.277660 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfad302-9aa2-46e5-80af-65b062d01cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:37.278286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfad302-9aa2-46e5-80af-65b062d01cec None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:37.278569 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfad302-9aa2-46e5-80af-65b062d01cec None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:37.278846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfad302-9aa2-46e5-80af-65b062d01cec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:37.301355 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:37.312287 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e6074-bc55-491f-a744-328edb9ce036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:37.312899 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e6074-bc55-491f-a744-328edb9ce036 None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:37.313184 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e6074-bc55-491f-a744-328edb9ce036 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:37.313457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e6074-bc55-491f-a744-328edb9ce036 None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:37.314167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e6074-bc55-491f-a744-328edb9ce036 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:37.315618 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e6074-bc55-491f-a744-328edb9ce036 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:37.337243 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 44/83] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Aug 4 04:25:37 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:25:37.337433 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:37.337962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e6074-bc55-491f-a744-328edb9ce036 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:37.359197 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfad302-9aa2-46e5-80af-65b062d01cec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:37.360731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfad302-9aa2-46e5-80af-65b062d01cec None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:37.457411 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e6074-bc55-491f-a744-328edb9ce036 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:37.486216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 40/84] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:25:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:25:37.487631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfad302-9aa2-46e5-80af-65b062d01cec None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:37.496670 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e9dae-3c65-4a08-bd20-d711ecaef4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:37.497236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e9dae-3c65-4a08-bd20-d711ecaef4fd None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:37.497522 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e9dae-3c65-4a08-bd20-d711ecaef4fd None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:37.497831 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e9dae-3c65-4a08-bd20-d711ecaef4fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:37.562150 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:37.599486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e9dae-3c65-4a08-bd20-d711ecaef4fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:37.602277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e9dae-3c65-4a08-bd20-d711ecaef4fd None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:37.607113 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:37.621517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfad302-9aa2-46e5-80af-65b062d01cec None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:37.666893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc90bf-fa0a-4ffd-8274-c309c027fcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:37.667524 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc90bf-fa0a-4ffd-8274-c309c027fcad None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:37.667813 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc90bf-fa0a-4ffd-8274-c309c027fcad None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:37.668090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc90bf-fa0a-4ffd-8274-c309c027fcad None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:37.669271 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc90bf-fa0a-4ffd-8274-c309c027fcad None admin] RBAC: Adding query filter params (name=member) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:37.669647 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc90bf-fa0a-4ffd-8274-c309c027fcad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:37.671042 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc90bf-fa0a-4ffd-8274-c309c027fcad None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:37.682180 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 41/85] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:25:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:37.686909 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc90bf-fa0a-4ffd-8274-c309c027fcad None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:37.732501 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 45/86] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:25:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:25:37.737225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e9dae-3c65-4a08-bd20-d711ecaef4fd None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:37.815606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc90bf-fa0a-4ffd-8274-c309c027fcad None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:37.836070 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:37.863669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1538078-e677-4c62-9fb7-d77c6fa76e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:37.864419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1538078-e677-4c62-9fb7-d77c6fa76e46 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:37.864779 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1538078-e677-4c62-9fb7-d77c6fa76e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:37.865220 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1538078-e677-4c62-9fb7-d77c6fa76e46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:37.865490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e9dae-3c65-4a08-bd20-d711ecaef4fd None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:37.893637 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:37.909678 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52900b76-789c-48e0-ad8f-a076c83a87b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:37.910328 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52900b76-789c-48e0-ad8f-a076c83a87b8 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:37.910614 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52900b76-789c-48e0-ad8f-a076c83a87b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:37.910896 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52900b76-789c-48e0-ad8f-a076c83a87b8 None admin] PATH_INFO: `/v3/projects/ba9c9c811ec54130a5b53525b32708d8` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:37.912123 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52900b76-789c-48e0-ad8f-a076c83a87b8 None admin] RBAC: Authorizing `identity:get_project(project_id=ba9c9c811ec54130a5b53525b32708d8)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:37.913615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52900b76-789c-48e0-ad8f-a076c83a87b8 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:37.915621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 46/87] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:25:37 2022] GET /identity/v3/projects/ba9c9c811ec54130a5b53525b32708d8 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:37.916409 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52900b76-789c-48e0-ad8f-a076c83a87b8 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:37.949976 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1538078-e677-4c62-9fb7-d77c6fa76e46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:37.951533 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1538078-e677-4c62-9fb7-d77c6fa76e46 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:38.037206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52900b76-789c-48e0-ad8f-a076c83a87b8 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:38.086883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ca8f1-2fae-402a-8ed1-f2f0a7dd499f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:38.087648 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ca8f1-2fae-402a-8ed1-f2f0a7dd499f None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:38.087950 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ca8f1-2fae-402a-8ed1-f2f0a7dd499f None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:38.088237 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ca8f1-2fae-402a-8ed1-f2f0a7dd499f None admin] PATH_INFO: `/v3/users/e82fe3500b4c40718ae00567a41063b9` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:38.089000 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 42/88] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:25:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:25:38.089596 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ca8f1-2fae-402a-8ed1-f2f0a7dd499f None admin] RBAC: Authorizing `identity:get_user(user_id=e82fe3500b4c40718ae00567a41063b9)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:38.089794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1538078-e677-4c62-9fb7-d77c6fa76e46 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:38.091113 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ca8f1-2fae-402a-8ed1-f2f0a7dd499f None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:38.094081 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 47/89] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:37 2022] GET /identity/v3/users/e82fe3500b4c40718ae00567a41063b9 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:38.094869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ca8f1-2fae-402a-8ed1-f2f0a7dd499f None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:38.119853 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:38.152233 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:38.209143 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1538078-e677-4c62-9fb7-d77c6fa76e46 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:38.244863 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ca8f1-2fae-402a-8ed1-f2f0a7dd499f None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:38.245816 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59497d-6430-41aa-b97a-454439cbf851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:38.246590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59497d-6430-41aa-b97a-454439cbf851 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:38.246981 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59497d-6430-41aa-b97a-454439cbf851 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:38.247350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59497d-6430-41aa-b97a-454439cbf851 None admin] PATH_INFO: `/v3/projects/ba9c9c811ec54130a5b53525b32708d8` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:38.249469 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59497d-6430-41aa-b97a-454439cbf851 None admin] RBAC: Authorizing `identity:get_project(project_id=ba9c9c811ec54130a5b53525b32708d8)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:38.250234 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59497d-6430-41aa-b97a-454439cbf851 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:38.252006 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 43/90] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:25:38 2022] GET /identity/v3/projects/ba9c9c811ec54130a5b53525b32708d8 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:38.258167 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59497d-6430-41aa-b97a-454439cbf851 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:38.298444 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f580433-44c4-42f0-8178-fa692b2ff891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:38.299335 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f580433-44c4-42f0-8178-fa692b2ff891 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:38.299762 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f580433-44c4-42f0-8178-fa692b2ff891 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:38.300165 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f580433-44c4-42f0-8178-fa692b2ff891 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:38.301094 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f580433-44c4-42f0-8178-fa692b2ff891 None admin] RBAC: Adding query filter params (role.id=e733372bae6047be956ad703f83a480e, scope.project.id=ba9c9c811ec54130a5b53525b32708d8, user.id=e82fe3500b4c40718ae00567a41063b9) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:38.301539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f580433-44c4-42f0-8178-fa692b2ff891 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:38.304459 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f580433-44c4-42f0-8178-fa692b2ff891 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:38.317595 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 48/91] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:25:38 2022] GET /identity/v3/role_assignments?user.id=e82fe3500b4c40718ae00567a41063b9&scope.project.id=ba9c9c811ec54130a5b53525b32708d8&role.id=e733372bae6047be956ad703f83a480e => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:38.317595 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f580433-44c4-42f0-8178-fa692b2ff891 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:38.434285 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59497d-6430-41aa-b97a-454439cbf851 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:38.485483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:38.495197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:38.515082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03144ea-a84e-45e9-aee4-2457ff31f21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:38.515757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03144ea-a84e-45e9-aee4-2457ff31f21a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:38.516042 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03144ea-a84e-45e9-aee4-2457ff31f21a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:38.516320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03144ea-a84e-45e9-aee4-2457ff31f21a None admin] PATH_INFO: `/v3/projects/ba9c9c811ec54130a5b53525b32708d8` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:38.518744 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03144ea-a84e-45e9-aee4-2457ff31f21a None admin] RBAC: Authorizing `identity:get_project(project_id=ba9c9c811ec54130a5b53525b32708d8)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:38.520225 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03144ea-a84e-45e9-aee4-2457ff31f21a None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:38.526204 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 44/92] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:25:38 2022] GET /identity/v3/projects/ba9c9c811ec54130a5b53525b32708d8 => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:38.527141 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03144ea-a84e-45e9-aee4-2457ff31f21a None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:38.539558 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f580433-44c4-42f0-8178-fa692b2ff891 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:38.589287 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db89011-3922-4589-8d96-8e46742ba140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:38.589960 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db89011-3922-4589-8d96-8e46742ba140 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:38.590248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db89011-3922-4589-8d96-8e46742ba140 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:38.590533 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db89011-3922-4589-8d96-8e46742ba140 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:38.640492 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03144ea-a84e-45e9-aee4-2457ff31f21a None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:38.687699 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed0cfd-6dc5-4c96-834d-e072a4161819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:38.689941 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed0cfd-6dc5-4c96-834d-e072a4161819 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:38.690704 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed0cfd-6dc5-4c96-834d-e072a4161819 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:38.690991 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed0cfd-6dc5-4c96-834d-e072a4161819 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:38.695535 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed0cfd-6dc5-4c96-834d-e072a4161819 None admin] RBAC: Adding query filter params (role.id=7a855ec41b944458821d76509b481268, scope.project.id=ba9c9c811ec54130a5b53525b32708d8, user.id=e82fe3500b4c40718ae00567a41063b9) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:38.695911 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed0cfd-6dc5-4c96-834d-e072a4161819 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:38.697392 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed0cfd-6dc5-4c96-834d-e072a4161819 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:38.715293 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 45/93] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:25:38 2022] GET /identity/v3/role_assignments?user.id=e82fe3500b4c40718ae00567a41063b9&scope.project.id=ba9c9c811ec54130a5b53525b32708d8&role.id=7a855ec41b944458821d76509b481268 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:38.716158 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed0cfd-6dc5-4c96-834d-e072a4161819 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:38.724006 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db89011-3922-4589-8d96-8e46742ba140 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:38.727461 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db89011-3922-4589-8d96-8e46742ba140 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:38.828900 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:38.853625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 49/94] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:25:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:25:38.858992 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db89011-3922-4589-8d96-8e46742ba140 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:38.873942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:38.945862 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed0cfd-6dc5-4c96-834d-e072a4161819 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:38.997865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb73d65-09ab-4a2c-a249-4f8a21320288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:38.998907 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb73d65-09ab-4a2c-a249-4f8a21320288 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:38.999199 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb73d65-09ab-4a2c-a249-4f8a21320288 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:38.999478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb73d65-09ab-4a2c-a249-4f8a21320288 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:39.000498 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb73d65-09ab-4a2c-a249-4f8a21320288 None admin] RBAC: Adding query filter params (role.id=4a6517e5890a49718303007dac0756c7, scope.project.id=ba9c9c811ec54130a5b53525b32708d8, user.id=fa79725fac2c4a528df2b060ede4922c) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:39.000825 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb73d65-09ab-4a2c-a249-4f8a21320288 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:39.005566 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb73d65-09ab-4a2c-a249-4f8a21320288 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:39.009312 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 46/95] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:25:38 2022] GET /identity/v3/role_assignments?user.id=fa79725fac2c4a528df2b060ede4922c&scope.project.id=ba9c9c811ec54130a5b53525b32708d8&role.id=4a6517e5890a49718303007dac0756c7 => generated 259 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:39.115706 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db89011-3922-4589-8d96-8e46742ba140 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:39.167426 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9decdd6-79f6-409c-b87f-17d4ab745585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:39.168470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9decdd6-79f6-409c-b87f-17d4ab745585 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:39.168940 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9decdd6-79f6-409c-b87f-17d4ab745585 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:39.169426 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9decdd6-79f6-409c-b87f-17d4ab745585 None admin] PATH_INFO: `/v3/projects/98a0065105354673b254c32a32bb1f8d` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:39.170856 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9decdd6-79f6-409c-b87f-17d4ab745585 None admin] RBAC: Authorizing `identity:get_project(project_id=98a0065105354673b254c32a32bb1f8d)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:39.172757 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9decdd6-79f6-409c-b87f-17d4ab745585 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:39.174945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 50/96] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:25:38 2022] GET /identity/v3/projects/98a0065105354673b254c32a32bb1f8d => generated 312 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:25:39.180986 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb73d65-09ab-4a2c-a249-4f8a21320288 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:39.336248 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb73d65-09ab-4a2c-a249-4f8a21320288 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:39.363845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:39.386520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440a92b-9997-4c0e-be74-34cc76a4071b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:39.387522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440a92b-9997-4c0e-be74-34cc76a4071b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:39.387999 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440a92b-9997-4c0e-be74-34cc76a4071b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:39.388466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440a92b-9997-4c0e-be74-34cc76a4071b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:39.389468 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440a92b-9997-4c0e-be74-34cc76a4071b None admin] RBAC: Adding query filter params (role.id=e733372bae6047be956ad703f83a480e, scope.project.id=98a0065105354673b254c32a32bb1f8d, user.id=e82fe3500b4c40718ae00567a41063b9) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:39.390011 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440a92b-9997-4c0e-be74-34cc76a4071b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:39.391655 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440a92b-9997-4c0e-be74-34cc76a4071b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:39.402137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 47/97] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:25:39 2022] GET /identity/v3/role_assignments?user.id=e82fe3500b4c40718ae00567a41063b9&scope.project.id=98a0065105354673b254c32a32bb1f8d&role.id=e733372bae6047be956ad703f83a480e => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:39.953000 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb574f3-47a5-4c28-892f-79741af38fb5 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:39.953703 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb574f3-47a5-4c28-892f-79741af38fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:39.954185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb574f3-47a5-4c28-892f-79741af38fb5 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:39.955961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 51/98] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:39.961367 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076a130-fa59-498a-a630-c721e20c2b0f None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:39.961922 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076a130-fa59-498a-a630-c721e20c2b0f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:39.962391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076a130-fa59-498a-a630-c721e20c2b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:39.986515 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-0076a130-fa59-498a-a630-c721e20c2b0f None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:39.988017 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0076a130-fa59-498a-a630-c721e20c2b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:40.074864 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 48/99] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:40.115196 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb574f3-47a5-4c28-892f-79741af38fb5 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:40.162801 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f1930-a34c-4309-9a01-4bf099dd44f7 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:40.163480 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f1930-a34c-4309-9a01-4bf099dd44f7 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:40.163954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f1930-a34c-4309-9a01-4bf099dd44f7 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:40.165492 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 49/100] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:25:40.170642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9997fca-347d-4319-aecd-1db651bdf472 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:40.171163 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9997fca-347d-4319-aecd-1db651bdf472 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:40.171629 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9997fca-347d-4319-aecd-1db651bdf472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:40.196049 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-a9997fca-347d-4319-aecd-1db651bdf472 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:40.198273 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:40.199707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a9997fca-347d-4319-aecd-1db651bdf472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:40.298407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb574f3-47a5-4c28-892f-79741af38fb5 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:40.335109 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 50/101] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:25:40.354695 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdaab6b-d3ef-43ae-9525-9c118b3abfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:40.355916 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdaab6b-d3ef-43ae-9525-9c118b3abfc6 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:40.356388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdaab6b-d3ef-43ae-9525-9c118b3abfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:40.356859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdaab6b-d3ef-43ae-9525-9c118b3abfc6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:40.359010 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdaab6b-d3ef-43ae-9525-9c118b3abfc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:40.361679 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdaab6b-d3ef-43ae-9525-9c118b3abfc6 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:40.364751 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 52/102] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:25:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:40.371257 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9997fca-347d-4319-aecd-1db651bdf472 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:40.377286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdaab6b-d3ef-43ae-9525-9c118b3abfc6 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:40.461735 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:40.547474 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdaab6b-d3ef-43ae-9525-9c118b3abfc6 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:40.550145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9997fca-347d-4319-aecd-1db651bdf472 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:40.571072 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:40.586352 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b41c83-9028-4f71-8c0e-7a8189f6f7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:40.587330 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b41c83-9028-4f71-8c0e-7a8189f6f7f3 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:40.587823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b41c83-9028-4f71-8c0e-7a8189f6f7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:40.588305 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b41c83-9028-4f71-8c0e-7a8189f6f7f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:40.589193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b41c83-9028-4f71-8c0e-7a8189f6f7f3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:40.590873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b41c83-9028-4f71-8c0e-7a8189f6f7f3 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:40.613696 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f803da51-d82f-435c-9283-b76fb397f9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:40.614726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803da51-d82f-435c-9283-b76fb397f9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:40.615411 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803da51-d82f-435c-9283-b76fb397f9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:40.616424 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803da51-d82f-435c-9283-b76fb397f9d8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:40.635574 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f803da51-d82f-435c-9283-b76fb397f9d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:40.648326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-f803da51-d82f-435c-9283-b76fb397f9d8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 04:25:40.648326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:40.648326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:40.648326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:40.648326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:40.648326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:40.648326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:40.648326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:40.648326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:40.648326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:40.648326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:40.648326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:40.648326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:40.648326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:40.648326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:40.648326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:40.649228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:40.649228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:40.649228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:40.649228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:40.649228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:40.649228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:40.649228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:40.649228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:40.649228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:40.649228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:40.649228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:40.649228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:40.649228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:40.649228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:40.649228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:40.649228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:40.649986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:40.649986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:40.649986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:40.649986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:40.649986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:40.649986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:40.649986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:40.649986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:40.649986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:40.649986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:40.649986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 04:25:40.649986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:25:40.652605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 51/103] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Aug 4 04:25:40 2022] POST /identity/v3/projects => generated 302 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:25:40.654138 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 53/104] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:25:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:25:40.671105 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b41c83-9028-4f71-8c0e-7a8189f6f7f3 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:40.799850 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:40.810606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b41c83-9028-4f71-8c0e-7a8189f6f7f3 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:40.847944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2b64f-5c8d-493f-878b-c45de16de106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:40.848576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2b64f-5c8d-493f-878b-c45de16de106 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:40.848859 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2b64f-5c8d-493f-878b-c45de16de106 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:40.849131 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2b64f-5c8d-493f-878b-c45de16de106 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:40.850482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f2b64f-5c8d-493f-878b-c45de16de106 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:40.850869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f2b64f-5c8d-493f-878b-c45de16de106 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:40.852232 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f2b64f-5c8d-493f-878b-c45de16de106 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:40.861494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 52/105] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:25:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:40.868075 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f803da51-d82f-435c-9283-b76fb397f9d8 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:40.997135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f803da51-d82f-435c-9283-b76fb397f9d8 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:41.036911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77205bd4-ef22-4e19-a05f-b842a2972dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:41.038110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77205bd4-ef22-4e19-a05f-b842a2972dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:41.038393 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77205bd4-ef22-4e19-a05f-b842a2972dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:41.038666 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77205bd4-ef22-4e19-a05f-b842a2972dc2 None admin] PATH_INFO: `/v3/users/5742a22c7de24485ac0eebea1f726650` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:41.054513 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77205bd4-ef22-4e19-a05f-b842a2972dc2 None admin] RBAC: Authorizing `identity:get_user(user_id=5742a22c7de24485ac0eebea1f726650)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:41.055979 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77205bd4-ef22-4e19-a05f-b842a2972dc2 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:41.057985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 54/106] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:40 2022] GET /identity/v3/users/5742a22c7de24485ac0eebea1f726650 => generated 289 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:41.063551 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2b64f-5c8d-493f-878b-c45de16de106 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:41.172678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2b64f-5c8d-493f-878b-c45de16de106 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:41.208133 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17e73f-3b9c-45fa-988b-e7d199617e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:41.209446 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17e73f-3b9c-45fa-988b-e7d199617e8e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:41.209833 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17e73f-3b9c-45fa-988b-e7d199617e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:41.210121 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17e73f-3b9c-45fa-988b-e7d199617e8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:41.251681 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:41.283038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f17e73f-3b9c-45fa-988b-e7d199617e8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:41.284563 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f17e73f-3b9c-45fa-988b-e7d199617e8e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:41.307019 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eaf422-f2fc-40c7-abb0-a2f1b1b24a21 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:41.307551 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eaf422-f2fc-40c7-abb0-a2f1b1b24a21 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:41.307748 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eaf422-f2fc-40c7-abb0-a2f1b1b24a21 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:41.309495 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 55/107] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:41.314857 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a5827-174e-4eb9-a895-d8321964d4c5 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:41.315098 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a5827-174e-4eb9-a895-d8321964d4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:41.315395 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a5827-174e-4eb9-a895-d8321964d4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:41.347402 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-411a5827-174e-4eb9-a895-d8321964d4c5 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:41.348610 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-411a5827-174e-4eb9-a895-d8321964d4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:41.408719 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:41.465401 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 53/108] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:25:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:25:41.466449 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 56/109] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:25:41.471956 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17e73f-3b9c-45fa-988b-e7d199617e8e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:41.497822 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a5827-174e-4eb9-a895-d8321964d4c5 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:41.570142 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:41.648439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17e73f-3b9c-45fa-988b-e7d199617e8e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:41.670708 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a5827-174e-4eb9-a895-d8321964d4c5 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:41.721701 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250a0d5-e83f-4682-a841-205c5193032c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:41.722784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250a0d5-e83f-4682-a841-205c5193032c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:41.723257 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250a0d5-e83f-4682-a841-205c5193032c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:41.723738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250a0d5-e83f-4682-a841-205c5193032c None admin] PATH_INFO: `/v3/projects/ba9c9c811ec54130a5b53525b32708d8` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:41.725855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250a0d5-e83f-4682-a841-205c5193032c None admin] RBAC: Authorizing `identity:get_project(project_id=ba9c9c811ec54130a5b53525b32708d8)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:41.727559 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250a0d5-e83f-4682-a841-205c5193032c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:41.729487 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 54/110] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:25:41 2022] GET /identity/v3/projects/ba9c9c811ec54130a5b53525b32708d8 => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:25:41.735479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250a0d5-e83f-4682-a841-205c5193032c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:41.735889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0d57c-ce89-4582-9d6f-c06294d039c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:41.736664 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0d57c-ce89-4582-9d6f-c06294d039c1 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:41.736957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0d57c-ce89-4582-9d6f-c06294d039c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:41.737232 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0d57c-ce89-4582-9d6f-c06294d039c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:41.762526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0d57c-ce89-4582-9d6f-c06294d039c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:41.767604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-b5c0d57c-ce89-4582-9d6f-c06294d039c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:25:41.767604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:41.767604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:41.767604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:41.767604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:41.767604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:41.767604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:41.767604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:41.767604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:41.767604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:41.767604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:41.767604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:41.767604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:41.767604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:41.767604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:41.767604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:41.768487 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:41.768487 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:41.768487 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:41.768487 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:41.768487 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:41.768487 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:41.768487 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:41.768487 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:41.768487 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:41.768487 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:41.768487 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:41.768487 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:41.768487 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:41.768487 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:41.768487 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:41.768487 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:41.769219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:41.769219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:41.769219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:41.769219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:41.769219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:41.769219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:41.769219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:41.769219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:41.769219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:41.769219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:41.769219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:25:41.769219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:25:41.773650 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 57/111] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:25:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:25:41.782458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0d57c-ce89-4582-9d6f-c06294d039c1 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:41.854742 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:41.962463 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0d57c-ce89-4582-9d6f-c06294d039c1 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:41.987671 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250a0d5-e83f-4682-a841-205c5193032c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:42.008616 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ec9a0-c988-494c-9922-4baba2eeb3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:42.010453 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ec9a0-c988-494c-9922-4baba2eeb3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:42.010744 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ec9a0-c988-494c-9922-4baba2eeb3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:42.011020 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ec9a0-c988-494c-9922-4baba2eeb3a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:42.011753 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ec9a0-c988-494c-9922-4baba2eeb3a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:42.012059 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ec9a0-c988-494c-9922-4baba2eeb3a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:42.013446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ec9a0-c988-494c-9922-4baba2eeb3a2 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:42.036414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 58/112] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:25:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:25:42.037578 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:42.043241 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ec9a0-c988-494c-9922-4baba2eeb3a2 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:42.064160 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce784ad1-ff35-4dea-9e39-e09530e25b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:42.064789 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce784ad1-ff35-4dea-9e39-e09530e25b2c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:42.065074 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce784ad1-ff35-4dea-9e39-e09530e25b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:42.065353 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce784ad1-ff35-4dea-9e39-e09530e25b2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:42.066183 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce784ad1-ff35-4dea-9e39-e09530e25b2c None admin] RBAC: Adding query filter params (role.id=f8e6c4a0973a483dbdf8131bf851686a, scope.project.id=ba9c9c811ec54130a5b53525b32708d8, user.id=5742a22c7de24485ac0eebea1f726650) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:42.067178 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce784ad1-ff35-4dea-9e39-e09530e25b2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:42.069714 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce784ad1-ff35-4dea-9e39-e09530e25b2c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:42.078109 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 55/113] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:25:41 2022] GET /identity/v3/role_assignments?user.id=5742a22c7de24485ac0eebea1f726650&scope.project.id=ba9c9c811ec54130a5b53525b32708d8&role.id=f8e6c4a0973a483dbdf8131bf851686a => generated 259 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:42.118381 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6e746-dae1-482f-bf44-d0a7f66a2bc8 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:42.119442 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6e746-dae1-482f-bf44-d0a7f66a2bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:42.120256 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6e746-dae1-482f-bf44-d0a7f66a2bc8 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:42.122379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 56/114] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:42.127427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400a5cc-704f-4033-b9d8-74a638f62402 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:42.127994 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400a5cc-704f-4033-b9d8-74a638f62402 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:42.128465 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400a5cc-704f-4033-b9d8-74a638f62402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:42.163425 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-5400a5cc-704f-4033-b9d8-74a638f62402 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:42.165288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-5400a5cc-704f-4033-b9d8-74a638f62402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:42.245617 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:42.260965 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 57/115] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:42.272752 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ec9a0-c988-494c-9922-4baba2eeb3a2 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:42.295074 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400a5cc-704f-4033-b9d8-74a638f62402 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:42.327491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9982a5-b206-4f34-90f3-c167f6e0ce89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:42.328244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9982a5-b206-4f34-90f3-c167f6e0ce89 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:42.328450 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9982a5-b206-4f34-90f3-c167f6e0ce89 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:42.328733 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9982a5-b206-4f34-90f3-c167f6e0ce89 None admin] PATH_INFO: `/v3/users/e82fe3500b4c40718ae00567a41063b9` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:42.330008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9982a5-b206-4f34-90f3-c167f6e0ce89 None admin] RBAC: Authorizing `identity:get_user(user_id=e82fe3500b4c40718ae00567a41063b9)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:42.331484 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9982a5-b206-4f34-90f3-c167f6e0ce89 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:42.333188 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 59/116] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:42 2022] GET /identity/v3/users/e82fe3500b4c40718ae00567a41063b9 => generated 275 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:42.338982 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b6dd4-e7d2-4cce-a40b-ebbbd6b9c0bd None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:42.339269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b6dd4-e7d2-4cce-a40b-ebbbd6b9c0bd None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:42.339575 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b6dd4-e7d2-4cce-a40b-ebbbd6b9c0bd None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:42.342136 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 60/117] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:42.342960 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b6dd4-e7d2-4cce-a40b-ebbbd6b9c0bd None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:42.418715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400a5cc-704f-4033-b9d8-74a638f62402 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:42.455627 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:42.483962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b6dd4-e7d2-4cce-a40b-ebbbd6b9c0bd None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:42.494450 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ef0f9-89a8-4e33-8aa7-eacd1781c219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:42.495480 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ef0f9-89a8-4e33-8aa7-eacd1781c219 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:42.497440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ef0f9-89a8-4e33-8aa7-eacd1781c219 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:42.497944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ef0f9-89a8-4e33-8aa7-eacd1781c219 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:42.521802 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ef0f9-89a8-4e33-8aa7-eacd1781c219 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:42.532313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-d56ef0f9-89a8-4e33-8aa7-eacd1781c219 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:25:42.532313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:42.532313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:42.532313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:42.532313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:42.532313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:42.532313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:42.532313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:42.532313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:42.532313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:42.532313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:42.532313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:42.532313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:42.532313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:42.532313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:42.532313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:42.534045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:42.534045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:42.534045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:42.534045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:42.534045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:42.534045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:42.534045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:42.534045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:42.534045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:42.534045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:42.534045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:42.534045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:42.534045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:42.534045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:42.534045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:42.534045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:42.535556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:42.535556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:42.535556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:42.535556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:42.535556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:42.535556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:42.535556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:42.535556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:42.535556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:42.535556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:42.535556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:25:42.535556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:25:42.535556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 58/118] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:25:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:25:42.542035 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119444af-f038-4a5a-b457-919d9aaf44ab None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:42.542035 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119444af-f038-4a5a-b457-919d9aaf44ab None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:42.542035 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119444af-f038-4a5a-b457-919d9aaf44ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:42.560533 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b963c0-9cd8-4c26-867a-801eced5a952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:42.561227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b963c0-9cd8-4c26-867a-801eced5a952 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:42.561507 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b963c0-9cd8-4c26-867a-801eced5a952 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:42.561818 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b963c0-9cd8-4c26-867a-801eced5a952 None admin] PATH_INFO: `/v3/projects/ba9c9c811ec54130a5b53525b32708d8` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:42.563077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b963c0-9cd8-4c26-867a-801eced5a952 None admin] RBAC: Authorizing `identity:get_project(project_id=ba9c9c811ec54130a5b53525b32708d8)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:42.563359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-119444af-f038-4a5a-b457-919d9aaf44ab None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:42.564555 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b963c0-9cd8-4c26-867a-801eced5a952 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:42.564675 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-119444af-f038-4a5a-b457-919d9aaf44ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:42.566262 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 61/119] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:25:42 2022] GET /identity/v3/projects/ba9c9c811ec54130a5b53525b32708d8 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:42.567317 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b963c0-9cd8-4c26-867a-801eced5a952 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:42.600864 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:42.645532 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 59/120] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:42.647868 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123b190-342d-4bbe-9ae5-dbf3e590723e None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:42.648168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123b190-342d-4bbe-9ae5-dbf3e590723e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:42.648550 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123b190-342d-4bbe-9ae5-dbf3e590723e None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:42.649894 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 60/121] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:42.654989 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f123b190-342d-4bbe-9ae5-dbf3e590723e None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:42.701429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b963c0-9cd8-4c26-867a-801eced5a952 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:42.746281 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46bc7b-7ca9-4c5b-81ed-8855354565c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:42.746933 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46bc7b-7ca9-4c5b-81ed-8855354565c6 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:42.747216 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46bc7b-7ca9-4c5b-81ed-8855354565c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:42.747492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46bc7b-7ca9-4c5b-81ed-8855354565c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:42.748243 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc46bc7b-7ca9-4c5b-81ed-8855354565c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:42.748586 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc46bc7b-7ca9-4c5b-81ed-8855354565c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:42.750132 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc46bc7b-7ca9-4c5b-81ed-8855354565c6 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:42.761219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 62/122] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:25:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:42.762518 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fee0a-ec7d-47cc-88dd-4b56e66fc130 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:42.762835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fee0a-ec7d-47cc-88dd-4b56e66fc130 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:42.763120 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fee0a-ec7d-47cc-88dd-4b56e66fc130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:42.783394 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-fe9fee0a-ec7d-47cc-88dd-4b56e66fc130 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:42.784590 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-fe9fee0a-ec7d-47cc-88dd-4b56e66fc130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:42.790029 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f123b190-342d-4bbe-9ae5-dbf3e590723e None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:42.805928 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:42.838813 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f79534-9b5f-4759-bbec-281b8f733ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:42.840045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f79534-9b5f-4759-bbec-281b8f733ee1 None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:42.840401 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f79534-9b5f-4759-bbec-281b8f733ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:42.840757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f79534-9b5f-4759-bbec-281b8f733ee1 None admin] PATH_INFO: `/v3/projects/ba9c9c811ec54130a5b53525b32708d8/users/e82fe3500b4c40718ae00567a41063b9/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:42.843492 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f79534-9b5f-4759-bbec-281b8f733ee1 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba9c9c811ec54130a5b53525b32708d8, user_id=e82fe3500b4c40718ae00567a41063b9, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:42.845654 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f79534-9b5f-4759-bbec-281b8f733ee1 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:42.857922 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 63/123] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:42.858862 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9fee0a-ec7d-47cc-88dd-4b56e66fc130 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:42.876042 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 61/124] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Aug 4 04:25:42 2022] PUT /identity/v3/projects/ba9c9c811ec54130a5b53525b32708d8/users/e82fe3500b4c40718ae00567a41063b9/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:25:42.876897 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f79534-9b5f-4759-bbec-281b8f733ee1 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:42.893152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:43.021163 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f79534-9b5f-4759-bbec-281b8f733ee1 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:43.064996 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9ec5e-1c53-4058-8d6e-6df10a03aa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:43.066036 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9ec5e-1c53-4058-8d6e-6df10a03aa70 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:43.066483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9ec5e-1c53-4058-8d6e-6df10a03aa70 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:43.066916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9ec5e-1c53-4058-8d6e-6df10a03aa70 None admin] PATH_INFO: `/v3/users/fa79725fac2c4a528df2b060ede4922c` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:43.069124 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e9ec5e-1c53-4058-8d6e-6df10a03aa70 None admin] RBAC: Authorizing `identity:get_user(user_id=fa79725fac2c4a528df2b060ede4922c)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:43.070973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e9ec5e-1c53-4058-8d6e-6df10a03aa70 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:43.072809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 62/125] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:42 2022] GET /identity/v3/users/fa79725fac2c4a528df2b060ede4922c => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:43.073860 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9ec5e-1c53-4058-8d6e-6df10a03aa70 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:43.088288 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9fee0a-ec7d-47cc-88dd-4b56e66fc130 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:43.113877 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:43.132625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac58c0-f6c4-442a-84bf-70329cc97e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:43.133633 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac58c0-f6c4-442a-84bf-70329cc97e7c None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:43.134068 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac58c0-f6c4-442a-84bf-70329cc97e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:43.135362 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac58c0-f6c4-442a-84bf-70329cc97e7c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:43.155110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ac58c0-f6c4-442a-84bf-70329cc97e7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:43.165873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-75ac58c0-f6c4-442a-84bf-70329cc97e7c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:25:43.165873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:43.165873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:43.165873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:43.165873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:43.165873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:43.165873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:43.165873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:43.165873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:43.165873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:43.165873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:43.165873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:43.165873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:43.165873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:43.165873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:43.165873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:43.166820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:43.166820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:43.166820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:43.166820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:43.166820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:43.166820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:43.166820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:43.166820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:43.166820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:43.166820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:43.166820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:43.166820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:43.166820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:43.166820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:43.166820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:43.166820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:43.167592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:43.167592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:43.167592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:43.167592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:43.167592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:43.167592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:43.167592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:43.167592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:43.167592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:43.167592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:43.167592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:25:43.167592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:25:43.167592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 64/126] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Aug 4 04:25:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:25:43.168231 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac58c0-f6c4-442a-84bf-70329cc97e7c None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:43.199610 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9ec5e-1c53-4058-8d6e-6df10a03aa70 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:43.236394 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2c766-1356-48c4-96b3-b691a183f1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:43.237377 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2c766-1356-48c4-96b3-b691a183f1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:43.237897 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2c766-1356-48c4-96b3-b691a183f1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:43.238363 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2c766-1356-48c4-96b3-b691a183f1f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:43.249734 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b2c766-1356-48c4-96b3-b691a183f1f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:43.256984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-90b2c766-1356-48c4-96b3-b691a183f1f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:25:43.256984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:43.256984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:43.256984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:43.256984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:43.256984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:43.256984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:43.256984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:43.256984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:43.256984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:43.256984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:43.256984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:43.256984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:43.256984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:43.256984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:43.256984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:43.257879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:43.257879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:43.257879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:43.257879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:43.257879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:43.257879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:43.257879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:43.257879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:43.257879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:43.257879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:43.257879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:43.257879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:43.257879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:43.257879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:43.257879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:43.257879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:43.258642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:43.258642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:43.258642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:43.258642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:43.258642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:43.258642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:43.258642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:43.258642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:43.258642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:43.258642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:43.258642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:25:43.258642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:25:43.260979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 63/127] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:25:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:25:43.261986 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2c766-1356-48c4-96b3-b691a183f1f0 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:43.267854 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:43.302889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac58c0-f6c4-442a-84bf-70329cc97e7c None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:43.341243 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6bddca-0cdd-4990-8535-242c429d1f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:43.342117 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bddca-0cdd-4990-8535-242c429d1f73 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:43.342376 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bddca-0cdd-4990-8535-242c429d1f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:43.342755 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bddca-0cdd-4990-8535-242c429d1f73 None admin] PATH_INFO: `/v3/projects/ba9c9c811ec54130a5b53525b32708d8` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:43.344139 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6bddca-0cdd-4990-8535-242c429d1f73 None admin] RBAC: Authorizing `identity:get_project(project_id=ba9c9c811ec54130a5b53525b32708d8)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:43.345656 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6bddca-0cdd-4990-8535-242c429d1f73 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:43.347370 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 65/128] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:25:43 2022] GET /identity/v3/projects/ba9c9c811ec54130a5b53525b32708d8 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:43.348217 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6bddca-0cdd-4990-8535-242c429d1f73 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:43.369363 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:43.377274 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2c766-1356-48c4-96b3-b691a183f1f0 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:43.413906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f67526d-9794-4753-b4ae-39a95cf5469c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:43.414884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67526d-9794-4753-b4ae-39a95cf5469c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:43.415461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67526d-9794-4753-b4ae-39a95cf5469c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:43.415941 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67526d-9794-4753-b4ae-39a95cf5469c None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:43.416863 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f67526d-9794-4753-b4ae-39a95cf5469c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:43.417358 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f67526d-9794-4753-b4ae-39a95cf5469c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:43.418967 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f67526d-9794-4753-b4ae-39a95cf5469c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:43.428698 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 64/129] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:25:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:43.429835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f67526d-9794-4753-b4ae-39a95cf5469c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:43.458437 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6bddca-0cdd-4990-8535-242c429d1f73 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:43.496640 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7015180-e9c9-4020-a065-3ce45d63b302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:43.497413 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7015180-e9c9-4020-a065-3ce45d63b302 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:43.497659 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7015180-e9c9-4020-a065-3ce45d63b302 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:43.497972 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7015180-e9c9-4020-a065-3ce45d63b302 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:43.498785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7015180-e9c9-4020-a065-3ce45d63b302 None admin] RBAC: Adding query filter params (name=member) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:43.499099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7015180-e9c9-4020-a065-3ce45d63b302 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:43.502098 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7015180-e9c9-4020-a065-3ce45d63b302 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:43.509907 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 66/130] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:25:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:43.515280 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7015180-e9c9-4020-a065-3ce45d63b302 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:43.519211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:43.549320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f67526d-9794-4753-b4ae-39a95cf5469c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:43.586235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9cf0d-fabf-461d-ac11-b820f2bad943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:43.587270 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9cf0d-fabf-461d-ac11-b820f2bad943 None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:43.587754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9cf0d-fabf-461d-ac11-b820f2bad943 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:43.588229 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9cf0d-fabf-461d-ac11-b820f2bad943 None admin] PATH_INFO: `/v3/projects/ba9c9c811ec54130a5b53525b32708d8/users/fa79725fac2c4a528df2b060ede4922c/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:43.590542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9cf0d-fabf-461d-ac11-b820f2bad943 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba9c9c811ec54130a5b53525b32708d8, user_id=fa79725fac2c4a528df2b060ede4922c, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:43.592219 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9cf0d-fabf-461d-ac11-b820f2bad943 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:43.625015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 65/131] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Aug 4 04:25:43 2022] PUT /identity/v3/projects/ba9c9c811ec54130a5b53525b32708d8/users/fa79725fac2c4a528df2b060ede4922c/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:25:43.627594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7015180-e9c9-4020-a065-3ce45d63b302 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:43.628737 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9cf0d-fabf-461d-ac11-b820f2bad943 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:43.670528 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa20eec-1a49-4f18-b893-38f8ca87257e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:43.671202 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa20eec-1a49-4f18-b893-38f8ca87257e None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:43.671487 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa20eec-1a49-4f18-b893-38f8ca87257e None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:43.672042 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa20eec-1a49-4f18-b893-38f8ca87257e None admin] PATH_INFO: `/v3/users/e82fe3500b4c40718ae00567a41063b9` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:43.673352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa20eec-1a49-4f18-b893-38f8ca87257e None admin] RBAC: Authorizing `identity:get_user(user_id=e82fe3500b4c40718ae00567a41063b9)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:43.675066 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa20eec-1a49-4f18-b893-38f8ca87257e None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:43.676877 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 67/132] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:43 2022] GET /identity/v3/users/e82fe3500b4c40718ae00567a41063b9 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:43.682854 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa20eec-1a49-4f18-b893-38f8ca87257e None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:43.702278 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:43.769786 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:43.808339 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa20eec-1a49-4f18-b893-38f8ca87257e None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:43.847946 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9cf0d-fabf-461d-ac11-b820f2bad943 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:43.854766 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7b8e4e-254e-4415-9a23-738a6b39ead2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:43.855641 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b8e4e-254e-4415-9a23-738a6b39ead2 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:43.856076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b8e4e-254e-4415-9a23-738a6b39ead2 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:43.856492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b8e4e-254e-4415-9a23-738a6b39ead2 None admin] PATH_INFO: `/v3/projects/ba9c9c811ec54130a5b53525b32708d8` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:43.858052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7b8e4e-254e-4415-9a23-738a6b39ead2 None admin] RBAC: Authorizing `identity:get_project(project_id=ba9c9c811ec54130a5b53525b32708d8)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:43.859648 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7b8e4e-254e-4415-9a23-738a6b39ead2 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:43.861747 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 68/133] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:25:43 2022] GET /identity/v3/projects/ba9c9c811ec54130a5b53525b32708d8 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:25:43.866931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaeab6e-2379-40be-8a5a-bc07b16292b3 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:43.867442 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaeab6e-2379-40be-8a5a-bc07b16292b3 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:43.867878 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaeab6e-2379-40be-8a5a-bc07b16292b3 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:43.869532 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 69/134] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:43.872182 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaeab6e-2379-40be-8a5a-bc07b16292b3 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:43.888280 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88888c9-30f5-4618-9d27-52948de33378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:43.889060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88888c9-30f5-4618-9d27-52948de33378 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:43.889491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88888c9-30f5-4618-9d27-52948de33378 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:43.889930 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88888c9-30f5-4618-9d27-52948de33378 None admin] PATH_INFO: `/v3/users/e82fe3500b4c40718ae00567a41063b9` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:43.891652 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88888c9-30f5-4618-9d27-52948de33378 None admin] RBAC: Authorizing `identity:get_user(user_id=e82fe3500b4c40718ae00567a41063b9)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:43.893045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88888c9-30f5-4618-9d27-52948de33378 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:43.899717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 66/135] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:43 2022] GET /identity/v3/users/e82fe3500b4c40718ae00567a41063b9 => generated 275 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:43.899717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b8a81-842a-4248-b51b-a18ee8d45c09 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:43.899717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b8a81-842a-4248-b51b-a18ee8d45c09 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:43.899717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b8a81-842a-4248-b51b-a18ee8d45c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:43.918692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-153b8a81-842a-4248-b51b-a18ee8d45c09 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:43.920257 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-153b8a81-842a-4248-b51b-a18ee8d45c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:43.994065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 67/136] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:43.997392 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b8a81-842a-4248-b51b-a18ee8d45c09 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:44.005289 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaeab6e-2379-40be-8a5a-bc07b16292b3 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:44.014453 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:44.067725 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9cbedb-ff11-4867-90d0-ff87d642ee10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:44.068947 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9cbedb-ff11-4867-90d0-ff87d642ee10 None admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:44.069400 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9cbedb-ff11-4867-90d0-ff87d642ee10 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:44.069848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9cbedb-ff11-4867-90d0-ff87d642ee10 None admin] PATH_INFO: `/v3/projects/ba9c9c811ec54130a5b53525b32708d8/users/e82fe3500b4c40718ae00567a41063b9/roles/7a855ec41b944458821d76509b481268` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:44.075388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9cbedb-ff11-4867-90d0-ff87d642ee10 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba9c9c811ec54130a5b53525b32708d8, user_id=e82fe3500b4c40718ae00567a41063b9, role_id=7a855ec41b944458821d76509b481268)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:44.077007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9cbedb-ff11-4867-90d0-ff87d642ee10 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:44.105106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:44.111562 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b8a81-842a-4248-b51b-a18ee8d45c09 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:44.112334 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 70/137] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Aug 4 04:25:43 2022] PUT /identity/v3/projects/ba9c9c811ec54130a5b53525b32708d8/users/e82fe3500b4c40718ae00567a41063b9/roles/7a855ec41b944458821d76509b481268 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:25:44.113163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9cbedb-ff11-4867-90d0-ff87d642ee10 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:44.154269 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f02fa-119a-4a11-b78b-62e7e3cb7319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:44.155017 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f02fa-119a-4a11-b78b-62e7e3cb7319 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:44.155389 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f02fa-119a-4a11-b78b-62e7e3cb7319 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:44.155729 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f02fa-119a-4a11-b78b-62e7e3cb7319 None admin] PATH_INFO: `/v3/projects/98a0065105354673b254c32a32bb1f8d` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:44.157199 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f02fa-119a-4a11-b78b-62e7e3cb7319 None admin] RBAC: Authorizing `identity:get_project(project_id=98a0065105354673b254c32a32bb1f8d)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:44.158916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f02fa-119a-4a11-b78b-62e7e3cb7319 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:44.160718 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 68/138] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:25:43 2022] GET /identity/v3/projects/98a0065105354673b254c32a32bb1f8d => generated 312 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:44.167546 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f02fa-119a-4a11-b78b-62e7e3cb7319 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:44.265472 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:44.285095 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f02fa-119a-4a11-b78b-62e7e3cb7319 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:44.325587 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e16b5-c47a-410d-bc25-10c60685400c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:44.326353 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e16b5-c47a-410d-bc25-10c60685400c None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:44.326693 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e16b5-c47a-410d-bc25-10c60685400c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:44.326974 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e16b5-c47a-410d-bc25-10c60685400c None admin] PATH_INFO: `/v3/projects/98a0065105354673b254c32a32bb1f8d/users/e82fe3500b4c40718ae00567a41063b9/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:44.328928 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290e16b5-c47a-410d-bc25-10c60685400c None admin] RBAC: Authorizing `identity:create_grant(project_id=98a0065105354673b254c32a32bb1f8d, user_id=e82fe3500b4c40718ae00567a41063b9, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:44.330411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290e16b5-c47a-410d-bc25-10c60685400c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:44.347101 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 69/139] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Aug 4 04:25:44 2022] PUT /identity/v3/projects/98a0065105354673b254c32a32bb1f8d/users/e82fe3500b4c40718ae00567a41063b9/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:25:44.355338 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9cbedb-ff11-4867-90d0-ff87d642ee10 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:44.411546 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c13ced-6934-44d4-898d-91c105f187f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:44.412802 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c13ced-6934-44d4-898d-91c105f187f1 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:44.413088 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c13ced-6934-44d4-898d-91c105f187f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:44.413358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c13ced-6934-44d4-898d-91c105f187f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:44.417460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c13ced-6934-44d4-898d-91c105f187f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:44.419003 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c13ced-6934-44d4-898d-91c105f187f1 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:44.421068 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 71/140] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:25:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:44.426745 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e16b5-c47a-410d-bc25-10c60685400c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:44.587018 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e16b5-c47a-410d-bc25-10c60685400c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:44.635360 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fda1bf-e170-480a-b993-b00682f075a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:44.636408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fda1bf-e170-480a-b993-b00682f075a8 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:44.636894 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fda1bf-e170-480a-b993-b00682f075a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:44.637513 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fda1bf-e170-480a-b993-b00682f075a8 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:44.638543 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fda1bf-e170-480a-b993-b00682f075a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:44.641136 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fda1bf-e170-480a-b993-b00682f075a8 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:44.676913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:44.682508 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 70/141] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Aug 4 04:25:44 2022] POST /identity/v3/users => generated 283 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:25:45.054589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4743c84-b5a0-4e93-9a87-7840ee26ea78 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:45.054975 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4743c84-b5a0-4e93-9a87-7840ee26ea78 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:45.055166 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4743c84-b5a0-4e93-9a87-7840ee26ea78 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:45.056575 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 72/142] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:45.062546 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31793af9-5f8d-49e8-97c9-58add6f77e84 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:45.062837 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31793af9-5f8d-49e8-97c9-58add6f77e84 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:45.063125 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31793af9-5f8d-49e8-97c9-58add6f77e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:45.090629 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-31793af9-5f8d-49e8-97c9-58add6f77e84 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:45.091845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-31793af9-5f8d-49e8-97c9-58add6f77e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:45.182139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 71/143] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:45.229917 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4743c84-b5a0-4e93-9a87-7840ee26ea78 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:45.317609 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:45.361907 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4743c84-b5a0-4e93-9a87-7840ee26ea78 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:45.406601 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c8b795-9712-4075-9819-4557e6e8a284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:45.407252 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8b795-9712-4075-9819-4557e6e8a284 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:45.407591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8b795-9712-4075-9819-4557e6e8a284 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:45.407872 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8b795-9712-4075-9819-4557e6e8a284 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:45.426422 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c8b795-9712-4075-9819-4557e6e8a284 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:45.437636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-17c8b795-9712-4075-9819-4557e6e8a284 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 04:25:45.437636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:45.437636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:45.437636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:45.437636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:45.437636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:45.437636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:45.437636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:45.437636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:45.437636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:45.437636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:45.437636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:45.437636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:45.437636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:45.437636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:45.437636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:45.438800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:45.438800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:45.438800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:45.438800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:45.438800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:45.438800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:45.438800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:45.438800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:45.438800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:45.438800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:45.438800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:45.438800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:45.438800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:45.438800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:45.438800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:45.438800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:45.439717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:45.439717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:45.439717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:45.439717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:45.439717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:45.439717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:45.439717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:45.439717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:45.439717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:45.439717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:45.439717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 04:25:45.439717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:25:45.439717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 73/144] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:25:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:25:45.446288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31793af9-5f8d-49e8-97c9-58add6f77e84 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:45.578599 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31793af9-5f8d-49e8-97c9-58add6f77e84 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:45.622101 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:45.628510 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73cbc4-c318-48a3-8672-cd4592b0a52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:45.629624 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73cbc4-c318-48a3-8672-cd4592b0a52f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:45.630449 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73cbc4-c318-48a3-8672-cd4592b0a52f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:45.631385 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73cbc4-c318-48a3-8672-cd4592b0a52f None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:45.632537 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a73cbc4-c318-48a3-8672-cd4592b0a52f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:45.632857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a73cbc4-c318-48a3-8672-cd4592b0a52f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:45.634313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a73cbc4-c318-48a3-8672-cd4592b0a52f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:45.644644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 72/145] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:25:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:25:45.651196 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c8b795-9712-4075-9819-4557e6e8a284 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:45.794121 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c8b795-9712-4075-9819-4557e6e8a284 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:45.831004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:45.840815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def7cc8-d8b4-4c09-b362-7a2be6b803ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:45.841536 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def7cc8-d8b4-4c09-b362-7a2be6b803ff None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:45.841868 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def7cc8-d8b4-4c09-b362-7a2be6b803ff None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:45.842289 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def7cc8-d8b4-4c09-b362-7a2be6b803ff None admin] PATH_INFO: `/v3/users/5742a22c7de24485ac0eebea1f726650` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:45.843502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def7cc8-d8b4-4c09-b362-7a2be6b803ff None admin] RBAC: Authorizing `identity:get_user(user_id=5742a22c7de24485ac0eebea1f726650)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:45.844974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def7cc8-d8b4-4c09-b362-7a2be6b803ff None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:45.846822 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 74/146] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:45 2022] GET /identity/v3/users/5742a22c7de24485ac0eebea1f726650 => generated 289 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:45.853116 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73cbc4-c318-48a3-8672-cd4592b0a52f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:45.996920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73cbc4-c318-48a3-8672-cd4592b0a52f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:46.037224 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375f79e-062b-4181-9639-37558ee3530c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:46.037897 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375f79e-062b-4181-9639-37558ee3530c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:46.038179 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375f79e-062b-4181-9639-37558ee3530c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:46.038458 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375f79e-062b-4181-9639-37558ee3530c None admin] PATH_INFO: `/v3/projects/ba9c9c811ec54130a5b53525b32708d8` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:46.040591 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7375f79e-062b-4181-9639-37558ee3530c None admin] RBAC: Authorizing `identity:get_project(project_id=ba9c9c811ec54130a5b53525b32708d8)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:46.042086 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7375f79e-062b-4181-9639-37558ee3530c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:46.044194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 73/147] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:25:45 2022] GET /identity/v3/projects/ba9c9c811ec54130a5b53525b32708d8 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:46.050038 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def7cc8-d8b4-4c09-b362-7a2be6b803ff None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:46.139999 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bccff-ea1d-437a-a758-ecf4c5a3bbdf None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:46.140291 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bccff-ea1d-437a-a758-ecf4c5a3bbdf None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:46.140562 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bccff-ea1d-437a-a758-ecf4c5a3bbdf None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:46.142166 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 74/148] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:46.149363 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23fb01-c3e3-4860-ab5e-063fe9f740f6 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:46.149715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23fb01-c3e3-4860-ab5e-063fe9f740f6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:46.149994 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23fb01-c3e3-4860-ab5e-063fe9f740f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:46.190634 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-4c23fb01-c3e3-4860-ab5e-063fe9f740f6 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:46.192514 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4c23fb01-c3e3-4860-ab5e-063fe9f740f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:46.198025 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def7cc8-d8b4-4c09-b362-7a2be6b803ff None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:46.263909 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:46.279731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 75/149] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:25:46.284592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec0cc0-0914-45f6-86c3-2005845f3821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:46.284956 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec0cc0-0914-45f6-86c3-2005845f3821 None admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:46.285244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec0cc0-0914-45f6-86c3-2005845f3821 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:46.285520 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec0cc0-0914-45f6-86c3-2005845f3821 None admin] PATH_INFO: `/v3/projects/ba9c9c811ec54130a5b53525b32708d8/users/5742a22c7de24485ac0eebea1f726650/roles/f8e6c4a0973a483dbdf8131bf851686a` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:46.290835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec0cc0-0914-45f6-86c3-2005845f3821 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba9c9c811ec54130a5b53525b32708d8, user_id=5742a22c7de24485ac0eebea1f726650, role_id=f8e6c4a0973a483dbdf8131bf851686a)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:46.292646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec0cc0-0914-45f6-86c3-2005845f3821 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:46.311161 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 75/150] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Aug 4 04:25:46 2022] PUT /identity/v3/projects/ba9c9c811ec54130a5b53525b32708d8/users/5742a22c7de24485ac0eebea1f726650/roles/f8e6c4a0973a483dbdf8131bf851686a => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:25:46.323832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c23fb01-c3e3-4860-ab5e-063fe9f740f6 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:46.445992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c23fb01-c3e3-4860-ab5e-063fe9f740f6 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:46.497787 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011e88d-e681-43d0-81a3-712d5db4990a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:46.506492 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011e88d-e681-43d0-81a3-712d5db4990a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:46.507021 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011e88d-e681-43d0-81a3-712d5db4990a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:46.507501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011e88d-e681-43d0-81a3-712d5db4990a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:46.531653 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011e88d-e681-43d0-81a3-712d5db4990a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:46.538077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-9011e88d-e681-43d0-81a3-712d5db4990a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:25:46.538077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:46.538077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:46.538077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:46.538077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:46.538077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:46.538077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:46.538077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:46.538077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:46.538077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:46.538077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:46.538077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:46.538077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:46.538077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:46.538077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:46.538077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:46.538910 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:46.538910 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:46.538910 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:46.538910 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:46.538910 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:46.538910 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:46.538910 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:46.538910 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:46.538910 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:46.538910 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:46.538910 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:46.538910 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:46.538910 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:46.538910 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:46.538910 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:46.538910 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:46.539644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:46.539644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:46.539644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:46.539644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:46.539644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:46.539644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:46.539644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:46.539644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:46.539644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:46.539644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:46.539644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:25:46.539644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:25:46.542067 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 76/151] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:25:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:25:46.554483 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec0cc0-0914-45f6-86c3-2005845f3821 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:46.669746 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:46.727219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec0cc0-0914-45f6-86c3-2005845f3821 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:46.768631 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c344a77-ac3b-41c4-9590-fcbb4fab143d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:46.769258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c344a77-ac3b-41c4-9590-fcbb4fab143d None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:46.769557 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c344a77-ac3b-41c4-9590-fcbb4fab143d None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:46.769846 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c344a77-ac3b-41c4-9590-fcbb4fab143d None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:46.770577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c344a77-ac3b-41c4-9590-fcbb4fab143d None admin] RBAC: Adding query filter params (name=member) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:46.770884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c344a77-ac3b-41c4-9590-fcbb4fab143d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:46.772720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c344a77-ac3b-41c4-9590-fcbb4fab143d None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:46.785800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 76/152] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:25:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:46.792484 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011e88d-e681-43d0-81a3-712d5db4990a None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:46.883172 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:46.945366 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011e88d-e681-43d0-81a3-712d5db4990a None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:46.987228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35dda8e-a109-49b7-914e-e49a01f2a1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:46.988326 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dda8e-a109-49b7-914e-e49a01f2a1fb None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:46.988799 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dda8e-a109-49b7-914e-e49a01f2a1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:46.989285 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dda8e-a109-49b7-914e-e49a01f2a1fb None admin] PATH_INFO: `/v3/users/e82fe3500b4c40718ae00567a41063b9` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:46.990787 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35dda8e-a109-49b7-914e-e49a01f2a1fb None admin] RBAC: Authorizing `identity:get_user(user_id=e82fe3500b4c40718ae00567a41063b9)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:46.992440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35dda8e-a109-49b7-914e-e49a01f2a1fb None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:46.994358 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 77/153] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:46 2022] GET /identity/v3/users/e82fe3500b4c40718ae00567a41063b9 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:47.000435 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c344a77-ac3b-41c4-9590-fcbb4fab143d None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:47.134413 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c344a77-ac3b-41c4-9590-fcbb4fab143d None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:47.144146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f698fd7-aba8-4d0f-abaf-16d6a9f945e6 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:47.144770 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f698fd7-aba8-4d0f-abaf-16d6a9f945e6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:47.145266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f698fd7-aba8-4d0f-abaf-16d6a9f945e6 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:47.147822 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 78/154] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:25:47.153378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64a4c3-6235-4a2a-b875-eed40c2b527e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:47.153968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64a4c3-6235-4a2a-b875-eed40c2b527e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:47.154441 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64a4c3-6235-4a2a-b875-eed40c2b527e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:47.160631 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:47.191698 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ef64a4c3-6235-4a2a-b875-eed40c2b527e None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:47.196233 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fceddab-064e-4b17-816c-0aaaf6ed3ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:47.196233 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fceddab-064e-4b17-816c-0aaaf6ed3ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:47.196233 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fceddab-064e-4b17-816c-0aaaf6ed3ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:47.196233 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fceddab-064e-4b17-816c-0aaaf6ed3ed0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:47.196964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ef64a4c3-6235-4a2a-b875-eed40c2b527e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:47.291384 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fceddab-064e-4b17-816c-0aaaf6ed3ed0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:47.292862 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fceddab-064e-4b17-816c-0aaaf6ed3ed0 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:47.300712 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 79/155] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:47.344500 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef64a4c3-6235-4a2a-b875-eed40c2b527e None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:47.443928 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:47.454714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 77/156] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:25:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:25:47.460573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fceddab-064e-4b17-816c-0aaaf6ed3ed0 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:47.493285 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:47.555699 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef64a4c3-6235-4a2a-b875-eed40c2b527e None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:47.615053 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103e4cc-6f95-4739-b34b-d3e9ce571444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:47.616061 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103e4cc-6f95-4739-b34b-d3e9ce571444 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:47.616667 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103e4cc-6f95-4739-b34b-d3e9ce571444 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:47.617150 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103e4cc-6f95-4739-b34b-d3e9ce571444 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:47.637227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5103e4cc-6f95-4739-b34b-d3e9ce571444 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:47.642613 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-5103e4cc-6f95-4739-b34b-d3e9ce571444 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:25:47.642613 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:47.642613 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:47.642613 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:47.642613 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:47.642613 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:47.642613 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:47.642613 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:47.642613 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:47.642613 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:47.642613 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:47.642613 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:47.642613 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:47.642613 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:47.642613 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:47.642613 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:47.643678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:47.643678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:47.643678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:47.643678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:47.643678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:47.643678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:47.643678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:47.643678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:47.643678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:47.643678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:47.643678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:47.643678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:47.643678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:47.643678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:47.643678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:47.643678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:47.644530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:47.644530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:47.644530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:47.644530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:47.644530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:47.644530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:47.644530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:47.644530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:47.644530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:47.644530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:47.644530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:25:47.644530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:25:47.647284 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 80/157] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:25:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:25:47.654817 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103e4cc-6f95-4739-b34b-d3e9ce571444 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:47.661362 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fceddab-064e-4b17-816c-0aaaf6ed3ed0 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:47.744168 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e22013-0b89-4efd-b222-39a10255fa45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:47.745190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e22013-0b89-4efd-b222-39a10255fa45 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:47.745924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e22013-0b89-4efd-b222-39a10255fa45 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:47.748179 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e22013-0b89-4efd-b222-39a10255fa45 None admin] PATH_INFO: `/v3/projects/ba9c9c811ec54130a5b53525b32708d8` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:47.749951 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e22013-0b89-4efd-b222-39a10255fa45 None admin] RBAC: Authorizing `identity:get_project(project_id=ba9c9c811ec54130a5b53525b32708d8)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:47.752216 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e22013-0b89-4efd-b222-39a10255fa45 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:47.754853 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 78/158] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:25:47 2022] GET /identity/v3/projects/ba9c9c811ec54130a5b53525b32708d8 => generated 298 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:25:47.761831 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e22013-0b89-4efd-b222-39a10255fa45 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:47.862204 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103e4cc-6f95-4739-b34b-d3e9ce571444 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:47.881899 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:47.909428 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:47.926996 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87497f-6bcd-499d-b9e3-2321fcebd400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:47.928557 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87497f-6bcd-499d-b9e3-2321fcebd400 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:47.929044 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87497f-6bcd-499d-b9e3-2321fcebd400 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:47.929526 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87497f-6bcd-499d-b9e3-2321fcebd400 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:47.930487 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87497f-6bcd-499d-b9e3-2321fcebd400 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:47.931045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87497f-6bcd-499d-b9e3-2321fcebd400 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:47.932630 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87497f-6bcd-499d-b9e3-2321fcebd400 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:47.937566 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e22013-0b89-4efd-b222-39a10255fa45 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:47.950045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 81/159] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:25:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:47.951752 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f30242-9f54-4bb6-93a4-ac87df26be46 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:47.952262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f30242-9f54-4bb6-93a4-ac87df26be46 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:47.952729 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f30242-9f54-4bb6-93a4-ac87df26be46 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:47.954407 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 82/160] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:47.960773 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97dc77a-a5ec-4cf5-8365-22bfe7617bbf None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:47.961467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97dc77a-a5ec-4cf5-8365-22bfe7617bbf None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:47.962156 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97dc77a-a5ec-4cf5-8365-22bfe7617bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:47.991673 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e169f3f-fbe0-4adc-92a8-01084608766d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:47.992293 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e169f3f-fbe0-4adc-92a8-01084608766d None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:47.992575 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e169f3f-fbe0-4adc-92a8-01084608766d None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:47.992849 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e169f3f-fbe0-4adc-92a8-01084608766d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:47.993680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e169f3f-fbe0-4adc-92a8-01084608766d None admin] RBAC: Adding query filter params (role.id=e733372bae6047be956ad703f83a480e, scope.project.id=ba9c9c811ec54130a5b53525b32708d8, user.id=e82fe3500b4c40718ae00567a41063b9) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:47.993988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e169f3f-fbe0-4adc-92a8-01084608766d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:47.995435 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e169f3f-fbe0-4adc-92a8-01084608766d None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:47.998542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-c97dc77a-a5ec-4cf5-8365-22bfe7617bbf None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:47.999931 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c97dc77a-a5ec-4cf5-8365-22bfe7617bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:48.006490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 79/161] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:25:47 2022] GET /identity/v3/role_assignments?user.id=e82fe3500b4c40718ae00567a41063b9&scope.project.id=ba9c9c811ec54130a5b53525b32708d8&role.id=e733372bae6047be956ad703f83a480e => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:48.007414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e169f3f-fbe0-4adc-92a8-01084608766d None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:48.165731 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e169f3f-fbe0-4adc-92a8-01084608766d None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:48.215318 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 83/162] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:48.216714 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afdb20-d430-45a0-aff5-bd7a5f311fd4 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:48.217009 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afdb20-d430-45a0-aff5-bd7a5f311fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:48.217286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afdb20-d430-45a0-aff5-bd7a5f311fd4 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:48.218820 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 84/163] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:48.229232 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d73a99-6ded-4397-b27f-9d8f96d45fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:48.229934 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d73a99-6ded-4397-b27f-9d8f96d45fcb None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:48.230220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d73a99-6ded-4397-b27f-9d8f96d45fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:48.230491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d73a99-6ded-4397-b27f-9d8f96d45fcb None admin] PATH_INFO: `/v3/users/fa79725fac2c4a528df2b060ede4922c` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:48.231724 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d73a99-6ded-4397-b27f-9d8f96d45fcb None admin] RBAC: Authorizing `identity:get_user(user_id=fa79725fac2c4a528df2b060ede4922c)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:48.233177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d73a99-6ded-4397-b27f-9d8f96d45fcb None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:48.234856 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 80/164] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:48 2022] GET /identity/v3/users/fa79725fac2c4a528df2b060ede4922c => generated 247 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:48.237216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffb1da-4ec5-4000-bc69-fb3f573d16d7 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:48.237502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffb1da-4ec5-4000-bc69-fb3f573d16d7 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:48.237818 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffb1da-4ec5-4000-bc69-fb3f573d16d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:48.242530 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d73a99-6ded-4397-b27f-9d8f96d45fcb None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:48.276738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:48.290718 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-33ffb1da-4ec5-4000-bc69-fb3f573d16d7 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:48.291948 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-33ffb1da-4ec5-4000-bc69-fb3f573d16d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:48.399576 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d73a99-6ded-4397-b27f-9d8f96d45fcb None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:48.463005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129858c0-68a9-4b32-890f-963fe4e548c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:48.463658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129858c0-68a9-4b32-890f-963fe4e548c5 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:48.463947 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129858c0-68a9-4b32-890f-963fe4e548c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:48.464221 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129858c0-68a9-4b32-890f-963fe4e548c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:48.467141 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 85/165] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:48.468099 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffb1da-4ec5-4000-bc69-fb3f573d16d7 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:48.523244 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:48.594180 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129858c0-68a9-4b32-890f-963fe4e548c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:48.595662 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129858c0-68a9-4b32-890f-963fe4e548c5 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:48.632927 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:48.650929 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffb1da-4ec5-4000-bc69-fb3f573d16d7 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:48.699207 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff15b7-6248-4931-9e85-84b716ac3846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:48.699893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff15b7-6248-4931-9e85-84b716ac3846 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:48.700173 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff15b7-6248-4931-9e85-84b716ac3846 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:48.700439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff15b7-6248-4931-9e85-84b716ac3846 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:48.712846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff15b7-6248-4931-9e85-84b716ac3846 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:48.720622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-d4ff15b7-6248-4931-9e85-84b716ac3846 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:25:48.720622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:48.720622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:48.720622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:48.720622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:48.720622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:48.720622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:48.720622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:48.720622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:48.720622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:48.720622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:48.720622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:48.720622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:48.720622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:48.720622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:48.720622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:48.721477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:48.721477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:48.721477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:48.721477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:48.721477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:48.721477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:48.721477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:48.721477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:48.721477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:48.721477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:48.721477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:48.721477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:48.721477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:48.721477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:48.721477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:48.721477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:48.722549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:48.722549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:48.722549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:48.722549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:48.722549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:48.722549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:48.722549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:48.722549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:48.722549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:48.722549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:48.722549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:25:48.722549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:25:48.722549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 86/166] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:25:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:25:48.722549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff15b7-6248-4931-9e85-84b716ac3846 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:48.794359 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 81/167] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:25:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:25:48.798305 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129858c0-68a9-4b32-890f-963fe4e548c5 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:48.925735 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:48.942143 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129858c0-68a9-4b32-890f-963fe4e548c5 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:48.962327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:48.967472 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff15b7-6248-4931-9e85-84b716ac3846 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:48.984691 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc508f-113b-42a9-9189-993b3fab4353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:48.985934 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc508f-113b-42a9-9189-993b3fab4353 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:48.986353 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc508f-113b-42a9-9189-993b3fab4353 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:48.986774 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc508f-113b-42a9-9189-993b3fab4353 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:48.987628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc508f-113b-42a9-9189-993b3fab4353 None admin] RBAC: Adding query filter params (name=member) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:48.988062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc508f-113b-42a9-9189-993b3fab4353 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:48.989627 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc508f-113b-42a9-9189-993b3fab4353 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:48.999353 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 82/168] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:25:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:49.000576 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc508f-113b-42a9-9189-993b3fab4353 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:49.070278 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bb961-8fc5-4d6e-9d71-bdc94818c5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:49.071193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bb961-8fc5-4d6e-9d71-bdc94818c5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:49.071193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bb961-8fc5-4d6e-9d71-bdc94818c5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:49.071476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bb961-8fc5-4d6e-9d71-bdc94818c5f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:49.110280 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bb961-8fc5-4d6e-9d71-bdc94818c5f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:49.116321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-bf7bb961-8fc5-4d6e-9d71-bdc94818c5f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:25:49.116321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:49.116321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:49.116321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:49.116321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:49.116321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:49.116321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:49.116321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:49.116321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:49.116321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:49.116321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:49.116321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:49.116321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:49.116321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:49.116321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:49.116321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:49.117410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:49.117410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:49.117410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:49.117410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:49.117410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:49.117410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:49.117410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:49.117410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:49.117410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:49.117410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:49.117410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:49.117410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:49.117410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:49.117410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:49.117410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:49.117410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:49.118308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:49.118308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:49.118308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:49.118308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:49.118308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:49.118308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:49.118308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:49.118308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:49.118308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:49.118308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:49.118308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:25:49.118308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:25:49.118308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 87/169] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Aug 4 04:25:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:25:49.118308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bb961-8fc5-4d6e-9d71-bdc94818c5f3 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:49.119558 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc508f-113b-42a9-9189-993b3fab4353 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:49.159174 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126fea51-51d9-4483-b278-4f58c46558c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:49.160079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126fea51-51d9-4483-b278-4f58c46558c3 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:49.160509 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126fea51-51d9-4483-b278-4f58c46558c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:49.160968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126fea51-51d9-4483-b278-4f58c46558c3 None admin] PATH_INFO: `/v3/projects/ba9c9c811ec54130a5b53525b32708d8` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:49.162982 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126fea51-51d9-4483-b278-4f58c46558c3 None admin] RBAC: Authorizing `identity:get_project(project_id=ba9c9c811ec54130a5b53525b32708d8)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:49.165186 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126fea51-51d9-4483-b278-4f58c46558c3 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:49.167399 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 83/170] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:25:48 2022] GET /identity/v3/projects/ba9c9c811ec54130a5b53525b32708d8 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:49.168395 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126fea51-51d9-4483-b278-4f58c46558c3 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:49.238575 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:49.318959 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126fea51-51d9-4483-b278-4f58c46558c3 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:49.423110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6ad91-97ce-46f1-82ff-345d5ba3a30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:49.423758 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6ad91-97ce-46f1-82ff-345d5ba3a30c None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:49.424038 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6ad91-97ce-46f1-82ff-345d5ba3a30c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:49.424312 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6ad91-97ce-46f1-82ff-345d5ba3a30c None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:49.425041 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6ad91-97ce-46f1-82ff-345d5ba3a30c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:49.425347 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6ad91-97ce-46f1-82ff-345d5ba3a30c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:49.426755 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6ad91-97ce-46f1-82ff-345d5ba3a30c None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:49.440599 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 84/171] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:25:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:49.441443 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6ad91-97ce-46f1-82ff-345d5ba3a30c None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:49.463752 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bb961-8fc5-4d6e-9d71-bdc94818c5f3 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:49.594455 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42a225-a198-4aea-9040-9ed824278c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:49.595126 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42a225-a198-4aea-9040-9ed824278c7e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:49.595419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42a225-a198-4aea-9040-9ed824278c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:49.595680 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42a225-a198-4aea-9040-9ed824278c7e None admin] PATH_INFO: `/v3/users/e82fe3500b4c40718ae00567a41063b9` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:49.596892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf42a225-a198-4aea-9040-9ed824278c7e None admin] RBAC: Authorizing `identity:get_user(user_id=e82fe3500b4c40718ae00567a41063b9)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:49.598387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf42a225-a198-4aea-9040-9ed824278c7e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:49.600592 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 88/172] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:49 2022] GET /identity/v3/users/e82fe3500b4c40718ae00567a41063b9 => generated 275 bytes in 483 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:49.603633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42a225-a198-4aea-9040-9ed824278c7e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:49.641918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:49.644966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6ad91-97ce-46f1-82ff-345d5ba3a30c None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:49.675011 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:49.709254 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb07a6a-97ac-48ba-add9-234b0138be71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:49.709929 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb07a6a-97ac-48ba-add9-234b0138be71 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:49.710212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb07a6a-97ac-48ba-add9-234b0138be71 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:49.710488 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb07a6a-97ac-48ba-add9-234b0138be71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:49.711634 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb07a6a-97ac-48ba-add9-234b0138be71 None admin] RBAC: Adding query filter params (role.id=4a6517e5890a49718303007dac0756c7, scope.project.id=ba9c9c811ec54130a5b53525b32708d8, user.id=fa79725fac2c4a528df2b060ede4922c) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:49.711955 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb07a6a-97ac-48ba-add9-234b0138be71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:49.713410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb07a6a-97ac-48ba-add9-234b0138be71 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:49.720531 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 85/173] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:25:49 2022] GET /identity/v3/role_assignments?user.id=fa79725fac2c4a528df2b060ede4922c&scope.project.id=ba9c9c811ec54130a5b53525b32708d8&role.id=4a6517e5890a49718303007dac0756c7 => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:49.722280 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef01e8a-75bd-4b7c-a875-42fa84397db9 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:49.722573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef01e8a-75bd-4b7c-a875-42fa84397db9 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:49.722856 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef01e8a-75bd-4b7c-a875-42fa84397db9 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:49.724850 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 86/174] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:49.725996 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef01e8a-75bd-4b7c-a875-42fa84397db9 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:49.848081 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef01e8a-75bd-4b7c-a875-42fa84397db9 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:49.879550 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42a225-a198-4aea-9040-9ed824278c7e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:49.888444 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5727d-1eaa-433a-abcf-4bc5cc32bdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:49.889080 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5727d-1eaa-433a-abcf-4bc5cc32bdce None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:49.889362 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5727d-1eaa-433a-abcf-4bc5cc32bdce None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:49.889663 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5727d-1eaa-433a-abcf-4bc5cc32bdce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:49.945097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:49.970641 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5727d-1eaa-433a-abcf-4bc5cc32bdce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:49.972391 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5727d-1eaa-433a-abcf-4bc5cc32bdce None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:49.985908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad6da9-c4e1-44ea-b06f-44b7a3286e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:49.986580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad6da9-c4e1-44ea-b06f-44b7a3286e64 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:49.986861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad6da9-c4e1-44ea-b06f-44b7a3286e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:49.987133 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad6da9-c4e1-44ea-b06f-44b7a3286e64 None admin] PATH_INFO: `/v3/users/e82fe3500b4c40718ae00567a41063b9` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:49.991919 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad6da9-c4e1-44ea-b06f-44b7a3286e64 None admin] RBAC: Authorizing `identity:get_user(user_id=e82fe3500b4c40718ae00567a41063b9)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:49.994251 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad6da9-c4e1-44ea-b06f-44b7a3286e64 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:49.996222 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 89/175] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:49 2022] GET /identity/v3/users/e82fe3500b4c40718ae00567a41063b9 => generated 275 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:49.997520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c67d2c-37dc-4c88-b160-922e04a54a95 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:49.997829 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c67d2c-37dc-4c88-b160-922e04a54a95 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:49.998105 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c67d2c-37dc-4c88-b160-922e04a54a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:50.026435 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-07c67d2c-37dc-4c88-b160-922e04a54a95 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:50.027754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-07c67d2c-37dc-4c88-b160-922e04a54a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:50.097573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 87/176] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:25:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:25:50.100237 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f38515-2a23-426b-ba02-30af703b7fff None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:50.100683 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f38515-2a23-426b-ba02-30af703b7fff None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:50.100972 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f38515-2a23-426b-ba02-30af703b7fff None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:50.102312 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 88/177] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:50.103085 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f38515-2a23-426b-ba02-30af703b7fff None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:50.118601 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 90/178] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:50.120026 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769ca1b-7d72-4dc9-b699-ad71f90ee2dd None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:50.120320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769ca1b-7d72-4dc9-b699-ad71f90ee2dd None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:50.120600 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769ca1b-7d72-4dc9-b699-ad71f90ee2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:50.141399 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-6769ca1b-7d72-4dc9-b699-ad71f90ee2dd None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:50.142775 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6769ca1b-7d72-4dc9-b699-ad71f90ee2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:50.161752 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:50.206385 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:50.223682 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 91/179] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:50.225572 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769ca1b-7d72-4dc9-b699-ad71f90ee2dd None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:50.235193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f38515-2a23-426b-ba02-30af703b7fff None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:50.270537 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60cee0-23b1-4020-ab3e-127b93453ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:50.271812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60cee0-23b1-4020-ab3e-127b93453ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:50.272302 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60cee0-23b1-4020-ab3e-127b93453ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:50.272585 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60cee0-23b1-4020-ab3e-127b93453ae3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:50.334652 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769ca1b-7d72-4dc9-b699-ad71f90ee2dd None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:50.341570 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d60cee0-23b1-4020-ab3e-127b93453ae3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:50.343049 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d60cee0-23b1-4020-ab3e-127b93453ae3 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:50.369650 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6bc6ed-e126-429d-ac80-564726ca937e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:50.370218 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bc6ed-e126-429d-ac80-564726ca937e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:50.370495 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bc6ed-e126-429d-ac80-564726ca937e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:50.370763 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bc6ed-e126-429d-ac80-564726ca937e None admin] PATH_INFO: `/v3/projects/98a0065105354673b254c32a32bb1f8d` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:50.371906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6bc6ed-e126-429d-ac80-564726ca937e None admin] RBAC: Authorizing `identity:get_project(project_id=98a0065105354673b254c32a32bb1f8d)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:50.373349 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6bc6ed-e126-429d-ac80-564726ca937e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:50.375013 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 92/180] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:25:50 2022] GET /identity/v3/projects/98a0065105354673b254c32a32bb1f8d => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:50.376115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e82bbb-dd04-40ea-b82a-08f095a348d2 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:50.376391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e82bbb-dd04-40ea-b82a-08f095a348d2 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:50.376663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e82bbb-dd04-40ea-b82a-08f095a348d2 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:50.377893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 93/181] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:50.378577 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e82bbb-dd04-40ea-b82a-08f095a348d2 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:50.394712 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:50.427342 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:50.453381 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 89/182] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:25:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:25:50.454694 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11dd44c-dd28-4462-88d2-273d9c09dee0 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:50.454977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11dd44c-dd28-4462-88d2-273d9c09dee0 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:50.455244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11dd44c-dd28-4462-88d2-273d9c09dee0 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:50.456482 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 90/183] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:50.457164 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11dd44c-dd28-4462-88d2-273d9c09dee0 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:50.505807 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e82bbb-dd04-40ea-b82a-08f095a348d2 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:50.543550 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231a25b-19dc-4138-b8b0-b37f5fa4c962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:50.544197 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231a25b-19dc-4138-b8b0-b37f5fa4c962 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:50.544479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231a25b-19dc-4138-b8b0-b37f5fa4c962 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:50.544773 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231a25b-19dc-4138-b8b0-b37f5fa4c962 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:50.555766 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f231a25b-19dc-4138-b8b0-b37f5fa4c962 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:50.561570 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11dd44c-dd28-4462-88d2-273d9c09dee0 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:50.561906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-f231a25b-19dc-4138-b8b0-b37f5fa4c962 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:25:50.561906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:50.561906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:50.561906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:50.561906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:50.561906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:50.561906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:50.561906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:50.561906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:50.561906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:50.561906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:50.561906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:50.561906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:50.561906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:50.561906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:50.561906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:50.562935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:50.562935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:50.562935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:50.562935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:50.562935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:50.562935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:50.562935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:50.562935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:50.562935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:50.562935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:50.562935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:50.562935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:50.562935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:50.562935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:50.562935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:50.562935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:50.563720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:50.563720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:50.563720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:50.563720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:50.563720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:50.563720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:50.563720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:50.563720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:50.563720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:50.563720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:50.563720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:25:50.563720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:25:50.563720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 94/184] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:25:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:25:50.564366 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231a25b-19dc-4138-b8b0-b37f5fa4c962 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:50.599443 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41c048-79d3-41fe-ac95-4f7a3c939f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:50.600086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41c048-79d3-41fe-ac95-4f7a3c939f1e None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:50.600367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41c048-79d3-41fe-ac95-4f7a3c939f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:50.600644 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41c048-79d3-41fe-ac95-4f7a3c939f1e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:50.601846 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c41c048-79d3-41fe-ac95-4f7a3c939f1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:50.603300 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c41c048-79d3-41fe-ac95-4f7a3c939f1e None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:50.604885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 91/185] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:25:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:50.609533 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcedea1-7402-4b07-840a-c2191238537a None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:50.609533 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcedea1-7402-4b07-840a-c2191238537a None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:50.609533 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcedea1-7402-4b07-840a-c2191238537a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:50.624544 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-6fcedea1-7402-4b07-840a-c2191238537a None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:50.625749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-6fcedea1-7402-4b07-840a-c2191238537a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:50.640974 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:50.662270 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:50.667583 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231a25b-19dc-4138-b8b0-b37f5fa4c962 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:50.695765 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 92/186] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:50.696764 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcedea1-7402-4b07-840a-c2191238537a None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:50.709363 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe9c73-be2f-4130-8c88-edbb90e1c976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:50.710362 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe9c73-be2f-4130-8c88-edbb90e1c976 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:50.710654 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe9c73-be2f-4130-8c88-edbb90e1c976 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:50.710927 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe9c73-be2f-4130-8c88-edbb90e1c976 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:50.712446 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbe9c73-be2f-4130-8c88-edbb90e1c976 None admin] RBAC: Adding query filter params (role.id=e733372bae6047be956ad703f83a480e, scope.project.id=98a0065105354673b254c32a32bb1f8d, user.id=e82fe3500b4c40718ae00567a41063b9) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:50.712773 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbe9c73-be2f-4130-8c88-edbb90e1c976 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:50.714809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbe9c73-be2f-4130-8c88-edbb90e1c976 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:50.721243 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 95/187] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:25:50 2022] GET /identity/v3/role_assignments?user.id=e82fe3500b4c40718ae00567a41063b9&scope.project.id=98a0065105354673b254c32a32bb1f8d&role.id=e733372bae6047be956ad703f83a480e => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:50.727011 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4153f6b-e982-44c8-9fde-68d108ecb3a8 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:50.727428 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4153f6b-e982-44c8-9fde-68d108ecb3a8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:50.727522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4153f6b-e982-44c8-9fde-68d108ecb3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:50.747133 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b4153f6b-e982-44c8-9fde-68d108ecb3a8 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:50.748364 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b4153f6b-e982-44c8-9fde-68d108ecb3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:50.823129 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcedea1-7402-4b07-840a-c2191238537a None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:50.841658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 96/188] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:50.841658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4153f6b-e982-44c8-9fde-68d108ecb3a8 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:50.873764 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4d582-d509-48a7-bc31-cc4541a599bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:50.874624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4d582-d509-48a7-bc31-cc4541a599bd None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:50.874890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4d582-d509-48a7-bc31-cc4541a599bd None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:50.875239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4d582-d509-48a7-bc31-cc4541a599bd None admin] PATH_INFO: `/v3/projects/ba9c9c811ec54130a5b53525b32708d8` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:50.876501 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4d582-d509-48a7-bc31-cc4541a599bd None admin] RBAC: Authorizing `identity:get_project(project_id=ba9c9c811ec54130a5b53525b32708d8)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:50.878057 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4d582-d509-48a7-bc31-cc4541a599bd None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:50.880166 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 93/189] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:25:50 2022] GET /identity/v3/projects/ba9c9c811ec54130a5b53525b32708d8 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:50.880645 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4d582-d509-48a7-bc31-cc4541a599bd None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:50.922880 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:50.930243 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:50.962613 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4153f6b-e982-44c8-9fde-68d108ecb3a8 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:50.991678 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4d582-d509-48a7-bc31-cc4541a599bd None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:50.996720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92b3ac-a991-435a-8e32-a5f88176f169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:50.997339 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92b3ac-a991-435a-8e32-a5f88176f169 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:50.997647 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92b3ac-a991-435a-8e32-a5f88176f169 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:50.997921 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92b3ac-a991-435a-8e32-a5f88176f169 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:50.998654 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92b3ac-a991-435a-8e32-a5f88176f169 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:50.998950 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92b3ac-a991-435a-8e32-a5f88176f169 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:51.000321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92b3ac-a991-435a-8e32-a5f88176f169 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:51.009409 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 97/190] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:25:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:51.010439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92b3ac-a991-435a-8e32-a5f88176f169 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:51.025908 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861031c-4460-491b-a75a-6cdec860a798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:51.026525 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861031c-4460-491b-a75a-6cdec860a798 None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:51.026804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861031c-4460-491b-a75a-6cdec860a798 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:51.027075 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861031c-4460-491b-a75a-6cdec860a798 None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:51.027815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861031c-4460-491b-a75a-6cdec860a798 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:51.029240 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861031c-4460-491b-a75a-6cdec860a798 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:51.049511 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 94/191] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Aug 4 04:25:50 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:25:51.059774 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861031c-4460-491b-a75a-6cdec860a798 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:51.127348 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92b3ac-a991-435a-8e32-a5f88176f169 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:51.167486 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:51.174243 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c1b32-2a84-497d-bc9d-ced4d9cf9185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:51.174897 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c1b32-2a84-497d-bc9d-ced4d9cf9185 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:51.175183 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c1b32-2a84-497d-bc9d-ced4d9cf9185 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:51.175460 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c1b32-2a84-497d-bc9d-ced4d9cf9185 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:51.176463 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:51.186286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861031c-4460-491b-a75a-6cdec860a798 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:51.187666 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458c1b32-2a84-497d-bc9d-ced4d9cf9185 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:51.194046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-458c1b32-2a84-497d-bc9d-ced4d9cf9185 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:25:51.194046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:51.194046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:51.194046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:51.194046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:51.194046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:51.194046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:51.194046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:51.194046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:51.194046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:51.194046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:51.194046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:51.194046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:51.194046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:51.194046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:51.194046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:51.194968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:51.194968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:51.194968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:51.194968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:51.194968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:51.194968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:51.194968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:51.194968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:51.194968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:51.194968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:51.194968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:51.194968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:51.194968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:51.194968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:51.194968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:51.194968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:51.195733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:51.195733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:51.195733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:51.195733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:51.195733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:51.195733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:51.195733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:51.195733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:51.195733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:51.195733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:51.195733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:25:51.195733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:25:51.195733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 98/192] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Aug 4 04:25:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:25:51.196355 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c1b32-2a84-497d-bc9d-ced4d9cf9185 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:51.234311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7ccda-e10e-4210-85d6-5677a44c3179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:51.234958 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7ccda-e10e-4210-85d6-5677a44c3179 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:51.235237 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7ccda-e10e-4210-85d6-5677a44c3179 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:51.235507 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7ccda-e10e-4210-85d6-5677a44c3179 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:51.246911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7ccda-e10e-4210-85d6-5677a44c3179 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:51.253594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-09b7ccda-e10e-4210-85d6-5677a44c3179 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 04:25:51.253594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:51.253594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:51.253594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:51.253594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:51.253594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:51.253594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:51.253594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:51.253594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:51.253594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:51.253594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:51.253594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:51.253594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:51.253594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:51.253594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:51.253594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:51.254476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:51.254476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:51.254476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:51.254476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:51.254476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:51.254476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:51.254476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:51.254476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:51.254476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:51.254476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:51.254476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:51.254476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:51.254476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:51.254476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:51.254476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:51.254476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:51.255852 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:51.255852 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:51.255852 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:51.255852 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:51.255852 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:51.255852 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:51.255852 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:51.255852 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:51.255852 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:51.255852 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:51.255852 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 04:25:51.255852 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:25:51.255852 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 95/193] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:25:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:25:51.255852 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7ccda-e10e-4210-85d6-5677a44c3179 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:51.340539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c1b32-2a84-497d-bc9d-ced4d9cf9185 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:51.383747 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3eb87-7ed6-41b4-a519-832d5c1c2738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:51.384329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3eb87-7ed6-41b4-a519-832d5c1c2738 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:51.384609 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3eb87-7ed6-41b4-a519-832d5c1c2738 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:51.384882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3eb87-7ed6-41b4-a519-832d5c1c2738 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:51.386084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c3eb87-7ed6-41b4-a519-832d5c1c2738 None admin] RBAC: Adding query filter params (role.id=7a855ec41b944458821d76509b481268, scope.project.id=ba9c9c811ec54130a5b53525b32708d8, user.id=e82fe3500b4c40718ae00567a41063b9) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:51.386400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c3eb87-7ed6-41b4-a519-832d5c1c2738 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:51.388423 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c3eb87-7ed6-41b4-a519-832d5c1c2738 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:51.390963 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7ccda-e10e-4210-85d6-5677a44c3179 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:51.396342 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 99/194] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:25:51 2022] GET /identity/v3/role_assignments?user.id=e82fe3500b4c40718ae00567a41063b9&scope.project.id=ba9c9c811ec54130a5b53525b32708d8&role.id=7a855ec41b944458821d76509b481268 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:51.397249 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3eb87-7ed6-41b4-a519-832d5c1c2738 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:51.435096 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb848742-217d-421b-94ed-59b33649622e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:51.435758 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb848742-217d-421b-94ed-59b33649622e None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:51.436033 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb848742-217d-421b-94ed-59b33649622e None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:51.436304 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb848742-217d-421b-94ed-59b33649622e None admin] PATH_INFO: `/v3/users/bd5f3d107ac94abcad21dfdd9b166ae7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:51.454353 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb848742-217d-421b-94ed-59b33649622e None admin] RBAC: Authorizing `identity:get_user(user_id=bd5f3d107ac94abcad21dfdd9b166ae7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:51.455904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb848742-217d-421b-94ed-59b33649622e None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:51.457825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 96/195] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:51 2022] GET /identity/v3/users/bd5f3d107ac94abcad21dfdd9b166ae7 => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:51.458506 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb848742-217d-421b-94ed-59b33649622e None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:51.477030 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:51.480506 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:51.568633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3eb87-7ed6-41b4-a519-832d5c1c2738 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:51.629929 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb848742-217d-421b-94ed-59b33649622e None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:51.631774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537bd6a-07df-4cc4-a059-772b246d6bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:51.632708 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537bd6a-07df-4cc4-a059-772b246d6bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:51.633178 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537bd6a-07df-4cc4-a059-772b246d6bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:51.633698 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537bd6a-07df-4cc4-a059-772b246d6bc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:51.634672 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537bd6a-07df-4cc4-a059-772b246d6bc5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:51.635169 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537bd6a-07df-4cc4-a059-772b246d6bc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:51.636776 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537bd6a-07df-4cc4-a059-772b246d6bc5 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:51.646894 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 100/196] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:25:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:51.648269 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537bd6a-07df-4cc4-a059-772b246d6bc5 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:51.675365 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4dfd2c-5103-44b7-85fb-28f90f0d8029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:51.676030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4dfd2c-5103-44b7-85fb-28f90f0d8029 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:51.676308 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4dfd2c-5103-44b7-85fb-28f90f0d8029 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:51.676635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4dfd2c-5103-44b7-85fb-28f90f0d8029 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:51.678001 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4dfd2c-5103-44b7-85fb-28f90f0d8029 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:51.678327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4dfd2c-5103-44b7-85fb-28f90f0d8029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:51.679681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4dfd2c-5103-44b7-85fb-28f90f0d8029 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:51.690301 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 97/197] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:25:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:51.691185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4dfd2c-5103-44b7-85fb-28f90f0d8029 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:51.772637 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537bd6a-07df-4cc4-a059-772b246d6bc5 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:51.790943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:51.793423 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:51.813117 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bec741-0a6e-4b85-b4c1-fbc1c4a7ee2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:51.814230 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bec741-0a6e-4b85-b4c1-fbc1c4a7ee2a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:51.814708 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bec741-0a6e-4b85-b4c1-fbc1c4a7ee2a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:51.815195 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bec741-0a6e-4b85-b4c1-fbc1c4a7ee2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:51.823264 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4dfd2c-5103-44b7-85fb-28f90f0d8029 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:51.862208 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5633ba-4d36-491c-a1eb-ed705f56125c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:51.862864 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5633ba-4d36-491c-a1eb-ed705f56125c None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:51.863147 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5633ba-4d36-491c-a1eb-ed705f56125c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:51.863424 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5633ba-4d36-491c-a1eb-ed705f56125c None admin] PATH_INFO: `/v3/users/bd5f3d107ac94abcad21dfdd9b166ae7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:51.864765 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5633ba-4d36-491c-a1eb-ed705f56125c None admin] RBAC: Authorizing `identity:get_user(user_id=bd5f3d107ac94abcad21dfdd9b166ae7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:51.866258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5633ba-4d36-491c-a1eb-ed705f56125c None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:51.868101 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 98/198] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:51 2022] GET /identity/v3/users/bd5f3d107ac94abcad21dfdd9b166ae7 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:51.869115 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5633ba-4d36-491c-a1eb-ed705f56125c None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:51.893283 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bec741-0a6e-4b85-b4c1-fbc1c4a7ee2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:51.895113 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bec741-0a6e-4b85-b4c1-fbc1c4a7ee2a None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:51.982821 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5633ba-4d36-491c-a1eb-ed705f56125c None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:52.006996 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 101/199] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:25:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:25:52.008514 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bec741-0a6e-4b85-b4c1-fbc1c4a7ee2a None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:52.021631 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ad372-b31e-46b0-8145-0a3b09c94cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:52.021631 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ad372-b31e-46b0-8145-0a3b09c94cfa None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:52.021887 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ad372-b31e-46b0-8145-0a3b09c94cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:52.022112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ad372-b31e-46b0-8145-0a3b09c94cfa None admin] PATH_INFO: `/v3/users/5742a22c7de24485ac0eebea1f726650` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:52.023334 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ad372-b31e-46b0-8145-0a3b09c94cfa None admin] RBAC: Authorizing `identity:get_user(user_id=5742a22c7de24485ac0eebea1f726650)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:52.024894 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ad372-b31e-46b0-8145-0a3b09c94cfa None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:52.026536 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 99/200] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:51 2022] GET /identity/v3/users/5742a22c7de24485ac0eebea1f726650 => generated 289 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:52.027381 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ad372-b31e-46b0-8145-0a3b09c94cfa None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:52.032645 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:52.045888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:52.122587 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bec741-0a6e-4b85-b4c1-fbc1c4a7ee2a None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:52.143122 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ad372-b31e-46b0-8145-0a3b09c94cfa None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:52.159712 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854b378-0b25-427f-a05b-9c949712e873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:52.160730 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854b378-0b25-427f-a05b-9c949712e873 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:52.161872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854b378-0b25-427f-a05b-9c949712e873 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:52.162466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854b378-0b25-427f-a05b-9c949712e873 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:52.179847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23f132-49dc-4d73-a100-c2bd3554033e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:52.180552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23f132-49dc-4d73-a100-c2bd3554033e None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:52.180904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23f132-49dc-4d73-a100-c2bd3554033e None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:52.181112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23f132-49dc-4d73-a100-c2bd3554033e None admin] PATH_INFO: `/v3/projects/88724aa044b04819920009d1b5332589` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:52.182312 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23f132-49dc-4d73-a100-c2bd3554033e None admin] RBAC: Authorizing `identity:get_project(project_id=88724aa044b04819920009d1b5332589)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:52.183971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23f132-49dc-4d73-a100-c2bd3554033e None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:52.185642 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 100/201] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:25:52 2022] GET /identity/v3/projects/88724aa044b04819920009d1b5332589 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:52.186619 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23f132-49dc-4d73-a100-c2bd3554033e None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:52.238903 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854b378-0b25-427f-a05b-9c949712e873 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:52.240895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854b378-0b25-427f-a05b-9c949712e873 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:52.258996 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:52.277356 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:52.298930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23f132-49dc-4d73-a100-c2bd3554033e None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:52.337339 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960b500-c1b6-4771-9df1-39b7f073dbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:52.338116 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960b500-c1b6-4771-9df1-39b7f073dbe2 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:52.338400 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960b500-c1b6-4771-9df1-39b7f073dbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:52.338675 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960b500-c1b6-4771-9df1-39b7f073dbe2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:52.351452 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 102/202] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:25:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:25:52.352669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854b378-0b25-427f-a05b-9c949712e873 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:52.415327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960b500-c1b6-4771-9df1-39b7f073dbe2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:52.416840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960b500-c1b6-4771-9df1-39b7f073dbe2 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:52.463702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854b378-0b25-427f-a05b-9c949712e873 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:52.504777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc8d0e-01b4-4e0b-ae8b-cb5f96d41aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:52.517573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc8d0e-01b4-4e0b-ae8b-cb5f96d41aaa None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:52.517573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:52.517573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc8d0e-01b4-4e0b-ae8b-cb5f96d41aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:52.517573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc8d0e-01b4-4e0b-ae8b-cb5f96d41aaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:52.517573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc8d0e-01b4-4e0b-ae8b-cb5f96d41aaa None admin] RBAC: Adding query filter params (role.id=4a6517e5890a49718303007dac0756c7, scope.project.id=88724aa044b04819920009d1b5332589, user.id=bd5f3d107ac94abcad21dfdd9b166ae7) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:52.517573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc8d0e-01b4-4e0b-ae8b-cb5f96d41aaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:52.518049 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc8d0e-01b4-4e0b-ae8b-cb5f96d41aaa None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:52.524820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:52.526663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 103/203] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:25:52 2022] GET /identity/v3/role_assignments?user.id=bd5f3d107ac94abcad21dfdd9b166ae7&scope.project.id=88724aa044b04819920009d1b5332589&role.id=4a6517e5890a49718303007dac0756c7 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:52.527672 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc8d0e-01b4-4e0b-ae8b-cb5f96d41aaa None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:52.539751 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 101/204] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:25:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:25:52.545788 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960b500-c1b6-4771-9df1-39b7f073dbe2 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:52.651542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc8d0e-01b4-4e0b-ae8b-cb5f96d41aaa None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:52.691542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6949afc-429c-40f2-a23d-736c87891f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:52.692264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6949afc-429c-40f2-a23d-736c87891f73 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:52.693111 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6949afc-429c-40f2-a23d-736c87891f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:52.693384 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6949afc-429c-40f2-a23d-736c87891f73 None admin] PATH_INFO: `/v3/projects/88724aa044b04819920009d1b5332589` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:52.694768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6949afc-429c-40f2-a23d-736c87891f73 None admin] RBAC: Authorizing `identity:get_project(project_id=88724aa044b04819920009d1b5332589)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:52.696305 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6949afc-429c-40f2-a23d-736c87891f73 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:52.698545 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 104/205] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:25:52 2022] GET /identity/v3/projects/88724aa044b04819920009d1b5332589 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:52.706612 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6949afc-429c-40f2-a23d-736c87891f73 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:52.782724 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960b500-c1b6-4771-9df1-39b7f073dbe2 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:52.795275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:52.821925 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33ce96-9c15-4262-b5c8-c4db0cbe3224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:52.822576 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33ce96-9c15-4262-b5c8-c4db0cbe3224 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:52.822856 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33ce96-9c15-4262-b5c8-c4db0cbe3224 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:52.823133 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33ce96-9c15-4262-b5c8-c4db0cbe3224 None admin] PATH_INFO: `/v3/projects/ba9c9c811ec54130a5b53525b32708d8` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:52.830338 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea33ce96-9c15-4262-b5c8-c4db0cbe3224 None admin] RBAC: Authorizing `identity:get_project(project_id=ba9c9c811ec54130a5b53525b32708d8)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:52.831788 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea33ce96-9c15-4262-b5c8-c4db0cbe3224 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:52.834123 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 102/206] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:25:52 2022] GET /identity/v3/projects/ba9c9c811ec54130a5b53525b32708d8 => generated 298 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:52.840150 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33ce96-9c15-4262-b5c8-c4db0cbe3224 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:52.875916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6949afc-429c-40f2-a23d-736c87891f73 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:52.893431 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:52.951929 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7c5ba-81b5-44c5-8955-effb5570a820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:52.952466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7c5ba-81b5-44c5-8955-effb5570a820 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:52.952751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7c5ba-81b5-44c5-8955-effb5570a820 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:52.953030 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7c5ba-81b5-44c5-8955-effb5570a820 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:52.953870 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7c5ba-81b5-44c5-8955-effb5570a820 None admin] RBAC: Adding query filter params (role.id=7a855ec41b944458821d76509b481268, scope.project.id=88724aa044b04819920009d1b5332589, user.id=bd5f3d107ac94abcad21dfdd9b166ae7) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:52.954183 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7c5ba-81b5-44c5-8955-effb5570a820 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:52.955629 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7c5ba-81b5-44c5-8955-effb5570a820 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:52.969068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 105/207] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:25:52 2022] GET /identity/v3/role_assignments?user.id=bd5f3d107ac94abcad21dfdd9b166ae7&scope.project.id=88724aa044b04819920009d1b5332589&role.id=7a855ec41b944458821d76509b481268 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:52.981776 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33ce96-9c15-4262-b5c8-c4db0cbe3224 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:53.019880 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c0bbb-a03e-4566-afd5-0ce6c75afbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:53.020498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c0bbb-a03e-4566-afd5-0ce6c75afbde None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:53.021346 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c0bbb-a03e-4566-afd5-0ce6c75afbde None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:53.021718 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c0bbb-a03e-4566-afd5-0ce6c75afbde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:53.022537 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c0bbb-a03e-4566-afd5-0ce6c75afbde None admin] RBAC: Adding query filter params (role.id=f8e6c4a0973a483dbdf8131bf851686a, scope.project.id=ba9c9c811ec54130a5b53525b32708d8, user.id=5742a22c7de24485ac0eebea1f726650) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:53.022847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c0bbb-a03e-4566-afd5-0ce6c75afbde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:53.024389 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c0bbb-a03e-4566-afd5-0ce6c75afbde None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:53.034490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 103/208] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:25:52 2022] GET /identity/v3/role_assignments?user.id=5742a22c7de24485ac0eebea1f726650&scope.project.id=ba9c9c811ec54130a5b53525b32708d8&role.id=f8e6c4a0973a483dbdf8131bf851686a => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:53.456066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146a8cd-6196-4c0f-81c4-cce167a69ae1 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:53.457061 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146a8cd-6196-4c0f-81c4-cce167a69ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:53.457585 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146a8cd-6196-4c0f-81c4-cce167a69ae1 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:53.459678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 106/209] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:53.465367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70925c1d-c612-4307-998a-09a14860a901 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:53.465747 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70925c1d-c612-4307-998a-09a14860a901 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:53.466032 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70925c1d-c612-4307-998a-09a14860a901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:53.492020 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-70925c1d-c612-4307-998a-09a14860a901 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:53.493231 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-70925c1d-c612-4307-998a-09a14860a901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:53.550964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2acb47-be09-4736-968f-fd78b6e49d14 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:53.551305 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2acb47-be09-4736-968f-fd78b6e49d14 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:53.551706 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2acb47-be09-4736-968f-fd78b6e49d14 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:53.553942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 107/210] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:53.558831 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631156d2-03f7-405c-86f7-817e8927a9ba None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:53.559113 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631156d2-03f7-405c-86f7-817e8927a9ba None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:53.559379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631156d2-03f7-405c-86f7-817e8927a9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:53.577450 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-631156d2-03f7-405c-86f7-817e8927a9ba None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:53.578700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 104/211] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:53.578958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-631156d2-03f7-405c-86f7-817e8927a9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:53.611298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70925c1d-c612-4307-998a-09a14860a901 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:53.661053 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 108/212] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:25:53.675482 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:53.696978 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631156d2-03f7-405c-86f7-817e8927a9ba None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:53.744549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:53.746524 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70925c1d-c612-4307-998a-09a14860a901 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:53.785160 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a4f04-bb9e-49d2-a4d0-c4e8a3baa71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:53.785855 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a4f04-bb9e-49d2-a4d0-c4e8a3baa71a None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:53.786429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a4f04-bb9e-49d2-a4d0-c4e8a3baa71a None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:53.786560 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a4f04-bb9e-49d2-a4d0-c4e8a3baa71a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:53.788363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a4f04-bb9e-49d2-a4d0-c4e8a3baa71a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:53.789110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a4f04-bb9e-49d2-a4d0-c4e8a3baa71a None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:53.791144 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 105/213] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:25:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:25:53.797458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a4f04-bb9e-49d2-a4d0-c4e8a3baa71a None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:53.831542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631156d2-03f7-405c-86f7-817e8927a9ba None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:53.878178 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bac34f-5b3e-4fa3-9d44-d18bbac88370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:53.878739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bac34f-5b3e-4fa3-9d44-d18bbac88370 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:53.879154 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bac34f-5b3e-4fa3-9d44-d18bbac88370 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:53.879529 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bac34f-5b3e-4fa3-9d44-d18bbac88370 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:53.891118 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bac34f-5b3e-4fa3-9d44-d18bbac88370 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:53.897479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-04bac34f-5b3e-4fa3-9d44-d18bbac88370 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:25:53.897479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:53.897479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:53.897479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:53.897479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:53.897479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:53.897479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:53.897479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:53.897479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:53.897479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:53.897479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:53.897479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:53.897479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:53.897479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:53.897479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:53.897479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:53.898576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:53.898576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:53.898576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:53.898576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:53.898576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:53.898576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:53.898576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:53.898576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:53.898576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:53.898576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:53.898576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:53.898576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:53.898576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:53.898576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:53.898576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:53.898576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:53.899512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:53.899512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:53.899512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:53.899512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:53.899512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:53.899512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:53.899512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:53.899512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:53.899512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:53.899512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:53.899512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:25:53.899512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:25:53.905827 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 109/214] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:25:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:25:53.908151 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bac34f-5b3e-4fa3-9d44-d18bbac88370 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:53.928816 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a4f04-bb9e-49d2-a4d0-c4e8a3baa71a None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:53.940972 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:53.968720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfe844-f112-4185-9f6e-cbbb1b67ce8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:53.969451 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfe844-f112-4185-9f6e-cbbb1b67ce8d None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:53.969857 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfe844-f112-4185-9f6e-cbbb1b67ce8d None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:53.970441 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfe844-f112-4185-9f6e-cbbb1b67ce8d None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:53.970968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dfe844-f112-4185-9f6e-cbbb1b67ce8d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:53.972485 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dfe844-f112-4185-9f6e-cbbb1b67ce8d None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:53.997996 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 106/215] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Aug 4 04:25:53 2022] POST /identity/v3/users => generated 297 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:25:54.051391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bac34f-5b3e-4fa3-9d44-d18bbac88370 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:54.054193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:54.097425 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb151d03-984c-4f76-9baf-55e8a0392360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:54.098412 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb151d03-984c-4f76-9baf-55e8a0392360 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:54.099780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb151d03-984c-4f76-9baf-55e8a0392360 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:54.100228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb151d03-984c-4f76-9baf-55e8a0392360 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:54.101212 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb151d03-984c-4f76-9baf-55e8a0392360 None admin] RBAC: Adding query filter params (name=member) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:54.101761 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb151d03-984c-4f76-9baf-55e8a0392360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:54.103405 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb151d03-984c-4f76-9baf-55e8a0392360 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:54.113188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 110/216] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:25:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:25:54.119384 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfe844-f112-4185-9f6e-cbbb1b67ce8d None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:54.232877 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfe844-f112-4185-9f6e-cbbb1b67ce8d None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:54.272195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a5c07-89b0-400e-9058-80ba34dfb159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:54.272875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a5c07-89b0-400e-9058-80ba34dfb159 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:54.273173 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a5c07-89b0-400e-9058-80ba34dfb159 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:54.273468 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a5c07-89b0-400e-9058-80ba34dfb159 None admin] PATH_INFO: `/v3/users/329048f276444e0da9cb3132aa915713` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:54.290708 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a5c07-89b0-400e-9058-80ba34dfb159 None admin] RBAC: Authorizing `identity:get_user(user_id=329048f276444e0da9cb3132aa915713)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:54.292472 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a5c07-89b0-400e-9058-80ba34dfb159 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:54.294533 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 107/217] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:54 2022] GET /identity/v3/users/329048f276444e0da9cb3132aa915713 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:54.300700 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb151d03-984c-4f76-9baf-55e8a0392360 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:54.423531 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb151d03-984c-4f76-9baf-55e8a0392360 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:54.458648 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08789c-4d68-4904-ab1a-577d7a98b34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:54.459368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08789c-4d68-4904-ab1a-577d7a98b34f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:54.459587 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08789c-4d68-4904-ab1a-577d7a98b34f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:54.459816 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08789c-4d68-4904-ab1a-577d7a98b34f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:54.501232 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:54.531887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08789c-4d68-4904-ab1a-577d7a98b34f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:54.533354 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08789c-4d68-4904-ab1a-577d7a98b34f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:54.638688 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 111/218] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:25:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:25:54.647069 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a5c07-89b0-400e-9058-80ba34dfb159 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:54.699340 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:54.805918 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a5c07-89b0-400e-9058-80ba34dfb159 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:54.850367 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cec98-7d5b-4afa-afa8-6bc4decd00d8 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:54.850706 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cec98-7d5b-4afa-afa8-6bc4decd00d8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:54.850923 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cec98-7d5b-4afa-afa8-6bc4decd00d8 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:54.856397 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 112/219] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:25:54.858349 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf0580-9c11-47f2-8718-6693ef2264d2 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:54.858674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf0580-9c11-47f2-8718-6693ef2264d2 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:54.859077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf0580-9c11-47f2-8718-6693ef2264d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:54.859395 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c0322-2cbc-47e8-8633-7d5d23dbe18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:54.860047 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c0322-2cbc-47e8-8633-7d5d23dbe18a None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:54.860328 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c0322-2cbc-47e8-8633-7d5d23dbe18a None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:54.860599 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c0322-2cbc-47e8-8633-7d5d23dbe18a None admin] PATH_INFO: `/v3/projects/88724aa044b04819920009d1b5332589` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:54.866339 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c0322-2cbc-47e8-8633-7d5d23dbe18a None admin] RBAC: Authorizing `identity:get_project(project_id=88724aa044b04819920009d1b5332589)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:54.867800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c0322-2cbc-47e8-8633-7d5d23dbe18a None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:54.869707 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 108/220] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:25:54 2022] GET /identity/v3/projects/88724aa044b04819920009d1b5332589 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:54.876430 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c0322-2cbc-47e8-8633-7d5d23dbe18a None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:54.877928 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-d9cf0580-9c11-47f2-8718-6693ef2264d2 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:54.879125 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d9cf0580-9c11-47f2-8718-6693ef2264d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:54.993948 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:54.998367 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 113/221] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:25:55.022494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf0580-9c11-47f2-8718-6693ef2264d2 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:55.036778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c0322-2cbc-47e8-8633-7d5d23dbe18a None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:55.045010 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:55.101978 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff94869-ec40-412e-92c7-81c50885c2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:55.102530 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff94869-ec40-412e-92c7-81c50885c2d1 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:55.102808 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff94869-ec40-412e-92c7-81c50885c2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:55.103082 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff94869-ec40-412e-92c7-81c50885c2d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:55.103883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff94869-ec40-412e-92c7-81c50885c2d1 None admin] RBAC: Adding query filter params (role.id=e733372bae6047be956ad703f83a480e, scope.project.id=88724aa044b04819920009d1b5332589, user.id=329048f276444e0da9cb3132aa915713) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:55.104193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff94869-ec40-412e-92c7-81c50885c2d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:55.105649 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff94869-ec40-412e-92c7-81c50885c2d1 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:55.115382 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 109/222] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:25:54 2022] GET /identity/v3/role_assignments?user.id=329048f276444e0da9cb3132aa915713&scope.project.id=88724aa044b04819920009d1b5332589&role.id=e733372bae6047be956ad703f83a480e => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:25:55.192811 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf0580-9c11-47f2-8718-6693ef2264d2 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:55.258406 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ddf7b3-4a4f-43c4-ace3-38d70abeeb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:55.259090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ddf7b3-4a4f-43c4-ace3-38d70abeeb7a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:55.259370 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ddf7b3-4a4f-43c4-ace3-38d70abeeb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:55.259644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ddf7b3-4a4f-43c4-ace3-38d70abeeb7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:55.275800 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ddf7b3-4a4f-43c4-ace3-38d70abeeb7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:55.281410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-93ddf7b3-4a4f-43c4-ace3-38d70abeeb7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:25:55.281410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:55.281410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:55.281410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:55.281410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:55.281410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:55.281410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:55.281410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:55.281410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:55.281410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:55.281410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:55.281410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:55.281410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:55.281410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:55.281410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:55.281410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:55.282320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:55.282320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:55.282320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:55.282320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:55.282320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:55.282320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:55.282320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:55.282320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:55.282320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:55.282320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:55.282320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:55.282320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:55.282320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:55.282320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:55.282320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:55.282320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:55.283090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:55.283090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:55.283090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:55.283090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:55.283090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:55.283090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:55.283090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:55.283090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:55.283090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:55.283090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:55.283090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:25:55.283090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:25:55.283090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 114/223] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:25:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:25:55.288980 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff94869-ec40-412e-92c7-81c50885c2d1 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:55.326346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59244e83-17fa-444a-82c3-2064d935471b None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:55.326640 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59244e83-17fa-444a-82c3-2064d935471b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:55.326906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59244e83-17fa-444a-82c3-2064d935471b None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:55.328256 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 115/224] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:25:55.333382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce303068-fa52-4300-a251-fdd0d171d16a None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:55.333927 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce303068-fa52-4300-a251-fdd0d171d16a None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:55.334849 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce303068-fa52-4300-a251-fdd0d171d16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:55.357425 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ce303068-fa52-4300-a251-fdd0d171d16a None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:55.359342 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ce303068-fa52-4300-a251-fdd0d171d16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:55.418756 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:55.442431 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 116/225] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:25:55.457116 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff94869-ec40-412e-92c7-81c50885c2d1 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:55.470317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce303068-fa52-4300-a251-fdd0d171d16a None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:55.517663 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92301204-e878-4d9c-918e-82b6f999f8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:55.518655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92301204-e878-4d9c-918e-82b6f999f8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:55.519142 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92301204-e878-4d9c-918e-82b6f999f8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:55.519695 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92301204-e878-4d9c-918e-82b6f999f8f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:55.521261 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92301204-e878-4d9c-918e-82b6f999f8f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:55.521798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92301204-e878-4d9c-918e-82b6f999f8f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:55.523367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92301204-e878-4d9c-918e-82b6f999f8f7 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:55.532590 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 110/226] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:25:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:55.539284 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92301204-e878-4d9c-918e-82b6f999f8f7 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:55.597962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:55.740107 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92301204-e878-4d9c-918e-82b6f999f8f7 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:55.760548 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce303068-fa52-4300-a251-fdd0d171d16a None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:55.815216 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82bde2-abef-4238-a691-ba24eae4aa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:55.816657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82bde2-abef-4238-a691-ba24eae4aa3f None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:55.818168 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82bde2-abef-4238-a691-ba24eae4aa3f None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:55.818773 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82bde2-abef-4238-a691-ba24eae4aa3f None admin] PATH_INFO: `/v3/users/bd5f3d107ac94abcad21dfdd9b166ae7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:55.821538 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e82bde2-abef-4238-a691-ba24eae4aa3f None admin] RBAC: Authorizing `identity:get_user(user_id=bd5f3d107ac94abcad21dfdd9b166ae7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:55.824895 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e82bde2-abef-4238-a691-ba24eae4aa3f None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:55.828843 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 111/227] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:55 2022] GET /identity/v3/users/bd5f3d107ac94abcad21dfdd9b166ae7 => generated 283 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:55.838846 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82bde2-abef-4238-a691-ba24eae4aa3f None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:55.844761 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764828aa-3c1e-4fbf-808f-a719de0d0545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:55.845611 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764828aa-3c1e-4fbf-808f-a719de0d0545 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:55.846047 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764828aa-3c1e-4fbf-808f-a719de0d0545 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:55.847244 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764828aa-3c1e-4fbf-808f-a719de0d0545 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:55.870193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764828aa-3c1e-4fbf-808f-a719de0d0545 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:55.875945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-764828aa-3c1e-4fbf-808f-a719de0d0545 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:25:55.875945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:55.875945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:55.875945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:55.875945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:55.875945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:55.875945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:55.875945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:55.875945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:55.875945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:55.875945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:55.875945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:55.875945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:55.875945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:55.875945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:55.875945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:55.877023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:55.877023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:55.877023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:55.877023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:55.877023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:55.877023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:55.877023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:55.877023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:55.877023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:55.877023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:55.877023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:55.877023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:55.877023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:55.877023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:55.877023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:55.877023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:55.877959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:55.877959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:55.877959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:55.877959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:55.877959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:55.877959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:55.877959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:55.877959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:55.877959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:55.877959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:55.877959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:25:55.877959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:25:55.880275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 117/228] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Aug 4 04:25:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:25:55.894217 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764828aa-3c1e-4fbf-808f-a719de0d0545 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:55.986061 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:56.022023 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:56.044840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82bde2-abef-4238-a691-ba24eae4aa3f None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:56.073384 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764828aa-3c1e-4fbf-808f-a719de0d0545 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:56.110191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3159544-acaa-4072-b154-ae63912aca12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:56.111227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3159544-acaa-4072-b154-ae63912aca12 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:56.111649 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3159544-acaa-4072-b154-ae63912aca12 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:56.112346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3159544-acaa-4072-b154-ae63912aca12 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:56.113488 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3159544-acaa-4072-b154-ae63912aca12 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:56.114693 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3159544-acaa-4072-b154-ae63912aca12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:56.117365 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3159544-acaa-4072-b154-ae63912aca12 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:56.123323 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8aa06-e156-4ae0-85ab-35fb70449da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:56.124085 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8aa06-e156-4ae0-85ab-35fb70449da8 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:56.124364 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8aa06-e156-4ae0-85ab-35fb70449da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:56.124641 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8aa06-e156-4ae0-85ab-35fb70449da8 None admin] PATH_INFO: `/v3/projects/88724aa044b04819920009d1b5332589` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:56.125854 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e8aa06-e156-4ae0-85ab-35fb70449da8 None admin] RBAC: Authorizing `identity:get_project(project_id=88724aa044b04819920009d1b5332589)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:56.127272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e8aa06-e156-4ae0-85ab-35fb70449da8 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:56.127881 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 118/229] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:25:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:56.132396 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 112/230] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:25:55 2022] GET /identity/v3/projects/88724aa044b04819920009d1b5332589 => generated 302 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:56.134795 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3159544-acaa-4072-b154-ae63912aca12 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:56.138825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8aa06-e156-4ae0-85ab-35fb70449da8 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:56.246961 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3159544-acaa-4072-b154-ae63912aca12 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:56.257913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:56.286606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4632598-dca3-4c70-a9b2-94914c61e7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:56.287545 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4632598-dca3-4c70-a9b2-94914c61e7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:56.288383 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4632598-dca3-4c70-a9b2-94914c61e7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:56.288832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4632598-dca3-4c70-a9b2-94914c61e7b2 None admin] PATH_INFO: `/v3/users/bd5f3d107ac94abcad21dfdd9b166ae7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:56.290381 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4632598-dca3-4c70-a9b2-94914c61e7b2 None admin] RBAC: Authorizing `identity:get_user(user_id=bd5f3d107ac94abcad21dfdd9b166ae7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:56.292156 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4632598-dca3-4c70-a9b2-94914c61e7b2 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:56.294208 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 119/231] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:56 2022] GET /identity/v3/users/bd5f3d107ac94abcad21dfdd9b166ae7 => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:25:56.300505 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4632598-dca3-4c70-a9b2-94914c61e7b2 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:56.413490 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4632598-dca3-4c70-a9b2-94914c61e7b2 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:56.415534 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8aa06-e156-4ae0-85ab-35fb70449da8 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:56.450021 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535eb90-0aa8-41c5-a8dc-9b87423718d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:56.451079 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535eb90-0aa8-41c5-a8dc-9b87423718d1 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:56.451560 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535eb90-0aa8-41c5-a8dc-9b87423718d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:56.452022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535eb90-0aa8-41c5-a8dc-9b87423718d1 None admin] PATH_INFO: `/v3/projects/88724aa044b04819920009d1b5332589` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:56.453514 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535eb90-0aa8-41c5-a8dc-9b87423718d1 None admin] RBAC: Authorizing `identity:get_project(project_id=88724aa044b04819920009d1b5332589)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:56.454989 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535eb90-0aa8-41c5-a8dc-9b87423718d1 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:56.456843 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 120/232] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:25:56 2022] GET /identity/v3/projects/88724aa044b04819920009d1b5332589 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:56.470001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535eb90-0aa8-41c5-a8dc-9b87423718d1 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:56.476490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2cad8-18c9-4334-b86f-170969563e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:56.477218 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2cad8-18c9-4334-b86f-170969563e12 None admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:56.477496 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2cad8-18c9-4334-b86f-170969563e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:56.477790 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2cad8-18c9-4334-b86f-170969563e12 None admin] PATH_INFO: `/v3/projects/88724aa044b04819920009d1b5332589/users/bd5f3d107ac94abcad21dfdd9b166ae7/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:56.480535 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2cad8-18c9-4334-b86f-170969563e12 None admin] RBAC: Authorizing `identity:create_grant(project_id=88724aa044b04819920009d1b5332589, user_id=bd5f3d107ac94abcad21dfdd9b166ae7, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:56.485530 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2cad8-18c9-4334-b86f-170969563e12 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:56.506411 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 113/233] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Aug 4 04:25:56 2022] PUT /identity/v3/projects/88724aa044b04819920009d1b5332589/users/bd5f3d107ac94abcad21dfdd9b166ae7/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:25:56.582247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:56.689274 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535eb90-0aa8-41c5-a8dc-9b87423718d1 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:56.734251 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76c294-739c-4601-89f0-72f238a7bc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:56.734251 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76c294-739c-4601-89f0-72f238a7bc9a None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:56.735530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76c294-739c-4601-89f0-72f238a7bc9a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:56.735530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76c294-739c-4601-89f0-72f238a7bc9a None admin] PATH_INFO: `/v3/projects/88724aa044b04819920009d1b5332589/users/bd5f3d107ac94abcad21dfdd9b166ae7/roles/7a855ec41b944458821d76509b481268` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:56.737713 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76c294-739c-4601-89f0-72f238a7bc9a None admin] RBAC: Authorizing `identity:create_grant(project_id=88724aa044b04819920009d1b5332589, user_id=bd5f3d107ac94abcad21dfdd9b166ae7, role_id=7a855ec41b944458821d76509b481268)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:56.739586 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76c294-739c-4601-89f0-72f238a7bc9a None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:56.753014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d48d4b-d966-46ed-b1f2-1c3485a00426 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:56.753295 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d48d4b-d966-46ed-b1f2-1c3485a00426 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:56.753594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d48d4b-d966-46ed-b1f2-1c3485a00426 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:56.755662 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 114/234] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:56.756083 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 121/235] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Aug 4 04:25:56 2022] PUT /identity/v3/projects/88724aa044b04819920009d1b5332589/users/bd5f3d107ac94abcad21dfdd9b166ae7/roles/7a855ec41b944458821d76509b481268 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:25:56.766081 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a7f666-f39e-4321-a950-4ae678b3d399 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:56.766399 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a7f666-f39e-4321-a950-4ae678b3d399 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:56.766685 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a7f666-f39e-4321-a950-4ae678b3d399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:56.800442 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-33a7f666-f39e-4321-a950-4ae678b3d399 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:56.802317 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-33a7f666-f39e-4321-a950-4ae678b3d399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:56.851176 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:56.920016 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 115/236] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:56.947945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76c294-739c-4601-89f0-72f238a7bc9a None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:57.059597 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76c294-739c-4601-89f0-72f238a7bc9a None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:57.062914 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:57.095812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37934b-ded1-499d-be6a-da981ef6927b None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:57.096134 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37934b-ded1-499d-be6a-da981ef6927b None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:57.096359 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37934b-ded1-499d-be6a-da981ef6927b None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:57.097801 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 116/237] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:25:57.104748 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c7d22-c3be-432e-874e-d799b73001ea None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:57.104986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c7d22-c3be-432e-874e-d799b73001ea None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:57.105240 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c7d22-c3be-432e-874e-d799b73001ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:57.127041 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b2eab-2ed3-4626-b4b5-d617c99a3f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:57.127729 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b2eab-2ed3-4626-b4b5-d617c99a3f5c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:57.128069 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b2eab-2ed3-4626-b4b5-d617c99a3f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:57.128368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b2eab-2ed3-4626-b4b5-d617c99a3f5c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:57.136856 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-608c7d22-c3be-432e-874e-d799b73001ea None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:57.138170 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-608c7d22-c3be-432e-874e-d799b73001ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:57.158465 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b2eab-2ed3-4626-b4b5-d617c99a3f5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:57.164110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-da0b2eab-2ed3-4626-b4b5-d617c99a3f5c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 04:25:57.164110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:57.164110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:57.164110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:57.164110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:57.164110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:57.164110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:57.164110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:57.164110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:57.164110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:57.164110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:57.164110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:57.164110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:57.164110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:57.164110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:57.164110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:57.164968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:57.164968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:57.164968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:57.164968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:57.164968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:57.164968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:57.164968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:57.164968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:57.164968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:57.164968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:57.164968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:57.164968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:57.164968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:57.164968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:57.164968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:57.164968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:57.165805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:57.165805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:57.165805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:57.165805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:57.165805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:57.165805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:57.165805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:57.165805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:57.165805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:57.165805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:57.165805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 04:25:57.165805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:25:57.168562 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 122/238] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:25:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:25:57.174625 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b2eab-2ed3-4626-b4b5-d617c99a3f5c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:57.244410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 117/239] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:57.297141 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c7d22-c3be-432e-874e-d799b73001ea None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:57.347033 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b2eab-2ed3-4626-b4b5-d617c99a3f5c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:57.421466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56a5a9-2400-44fd-a763-7423185ab2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:57.422305 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56a5a9-2400-44fd-a763-7423185ab2f6 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:57.422586 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56a5a9-2400-44fd-a763-7423185ab2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:57.422858 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56a5a9-2400-44fd-a763-7423185ab2f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:57.423596 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc56a5a9-2400-44fd-a763-7423185ab2f6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:57.423902 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc56a5a9-2400-44fd-a763-7423185ab2f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:57.426507 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc56a5a9-2400-44fd-a763-7423185ab2f6 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:57.440630 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:57.447695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 123/240] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:25:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:57.453634 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56a5a9-2400-44fd-a763-7423185ab2f6 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:57.481585 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:57.485808 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c7d22-c3be-432e-874e-d799b73001ea None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:57.554108 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b4c7f-411f-4326-8648-1f934aaa0ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:57.554770 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b4c7f-411f-4326-8648-1f934aaa0ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:57.555054 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b4c7f-411f-4326-8648-1f934aaa0ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:57.555329 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b4c7f-411f-4326-8648-1f934aaa0ac6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:57.556604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324b4c7f-411f-4326-8648-1f934aaa0ac6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:57.558343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324b4c7f-411f-4326-8648-1f934aaa0ac6 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:57.560615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 118/241] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:25:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:25:57.567985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56a5a9-2400-44fd-a763-7423185ab2f6 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:57.575928 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b4c7f-411f-4326-8648-1f934aaa0ac6 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:57.610438 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfeb76a-92ca-4d84-a199-9f31e9c0d7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:57.611089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfeb76a-92ca-4d84-a199-9f31e9c0d7f5 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:57.611353 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfeb76a-92ca-4d84-a199-9f31e9c0d7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:57.611625 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfeb76a-92ca-4d84-a199-9f31e9c0d7f5 None admin] PATH_INFO: `/v3/users/7b2310ed5e31441faf1790f8f3fe4dad` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:57.627427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfeb76a-92ca-4d84-a199-9f31e9c0d7f5 None admin] RBAC: Authorizing `identity:get_user(user_id=7b2310ed5e31441faf1790f8f3fe4dad)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:57.629015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfeb76a-92ca-4d84-a199-9f31e9c0d7f5 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:57.632267 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 124/242] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:57 2022] GET /identity/v3/users/7b2310ed5e31441faf1790f8f3fe4dad => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:57.642716 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfeb76a-92ca-4d84-a199-9f31e9c0d7f5 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:57.731258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b4c7f-411f-4326-8648-1f934aaa0ac6 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:57.768084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:57.778755 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9254a-391d-4014-9058-e984f6c14270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:57.779614 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9254a-391d-4014-9058-e984f6c14270 None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:57.780305 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9254a-391d-4014-9058-e984f6c14270 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:57.780581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9254a-391d-4014-9058-e984f6c14270 None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:57.781556 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9254a-391d-4014-9058-e984f6c14270 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:57.783832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9254a-391d-4014-9058-e984f6c14270 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:57.789700 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfeb76a-92ca-4d84-a199-9f31e9c0d7f5 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:57.807049 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:57.817238 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 119/243] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Aug 4 04:25:57 2022] POST /identity/v3/users => generated 293 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 04:25:57.818876 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c767105-eb9a-4a7d-8b4b-294137291d98 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:57.819398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c767105-eb9a-4a7d-8b4b-294137291d98 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:57.819942 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c767105-eb9a-4a7d-8b4b-294137291d98 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:57.821458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 120/244] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:57.827028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f41729f-b328-4117-9ad4-0e389a60d58c None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:57.827638 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f41729f-b328-4117-9ad4-0e389a60d58c None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:57.828148 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f41729f-b328-4117-9ad4-0e389a60d58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:57.851455 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-1f41729f-b328-4117-9ad4-0e389a60d58c None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:57.852082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa041aab-d545-466b-a6df-8c52b41257c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:57.852718 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa041aab-d545-466b-a6df-8c52b41257c0 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:57.852999 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa041aab-d545-466b-a6df-8c52b41257c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:57.853275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa041aab-d545-466b-a6df-8c52b41257c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:57.854143 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-1f41729f-b328-4117-9ad4-0e389a60d58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:57.934690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 121/245] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:57.968941 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f41729f-b328-4117-9ad4-0e389a60d58c None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:58.009722 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa041aab-d545-466b-a6df-8c52b41257c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:58.011627 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa041aab-d545-466b-a6df-8c52b41257c0 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:58.098191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f41729f-b328-4117-9ad4-0e389a60d58c None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:58.126905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:58.133908 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:58.139633 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff513e9-4f84-4d50-86d9-90ccecc36917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:58.140384 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff513e9-4f84-4d50-86d9-90ccecc36917 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:58.140824 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff513e9-4f84-4d50-86d9-90ccecc36917 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:58.141157 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff513e9-4f84-4d50-86d9-90ccecc36917 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:58.149060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 125/246] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:25:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 04:25:58.154506 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff513e9-4f84-4d50-86d9-90ccecc36917 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:58.154817 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa041aab-d545-466b-a6df-8c52b41257c0 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:58.160021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-1ff513e9-4f84-4d50-86d9-90ccecc36917 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:25:58.160021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:58.160021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:58.160021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:58.160021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:58.160021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:58.160021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:58.160021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:58.160021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:58.160021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:58.160021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:58.160021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:58.160021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:58.160021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:58.160021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:58.160021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:58.161655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:58.161655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:58.161655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:58.161655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:58.161655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:58.161655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:58.161655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:58.161655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:58.161655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:58.161655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:58.161655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:58.161655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:58.161655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:58.161655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:58.161655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:58.161655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:58.162825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:58.162825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:58.162825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:58.162825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:58.162825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:58.162825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:58.162825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:58.162825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:58.162825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:58.162825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:58.162825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:25:58.162825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:25:58.165459 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 122/247] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:25:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:25:58.171444 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff513e9-4f84-4d50-86d9-90ccecc36917 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:58.271617 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa041aab-d545-466b-a6df-8c52b41257c0 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:58.285893 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff513e9-4f84-4d50-86d9-90ccecc36917 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:58.312943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81db1a-96d2-4846-b79a-c15c4969348f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:58.313914 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81db1a-96d2-4846-b79a-c15c4969348f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:58.314193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81db1a-96d2-4846-b79a-c15c4969348f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:58.314471 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81db1a-96d2-4846-b79a-c15c4969348f None admin] PATH_INFO: `/v3/projects/88724aa044b04819920009d1b5332589` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:58.316322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81db1a-96d2-4846-b79a-c15c4969348f None admin] RBAC: Authorizing `identity:get_project(project_id=88724aa044b04819920009d1b5332589)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:58.318507 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81db1a-96d2-4846-b79a-c15c4969348f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:58.325272 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 126/248] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:25:58 2022] GET /identity/v3/projects/88724aa044b04819920009d1b5332589 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:58.327803 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81db1a-96d2-4846-b79a-c15c4969348f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:58.330913 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60580533-8a29-4571-b37c-7e964b87db3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:58.331646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60580533-8a29-4571-b37c-7e964b87db3d None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:58.331933 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60580533-8a29-4571-b37c-7e964b87db3d None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:58.332211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60580533-8a29-4571-b37c-7e964b87db3d None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:58.333470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60580533-8a29-4571-b37c-7e964b87db3d None admin] RBAC: Adding query filter params (name=member) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:58.333825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60580533-8a29-4571-b37c-7e964b87db3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:58.335215 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60580533-8a29-4571-b37c-7e964b87db3d None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:58.349308 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 123/249] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:25:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:25:58.354745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60580533-8a29-4571-b37c-7e964b87db3d None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:58.394014 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:58.433316 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:58.463108 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81db1a-96d2-4846-b79a-c15c4969348f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:58.514450 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d2b0a-593b-4af1-a6a8-19d4962e8942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:58.515148 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d2b0a-593b-4af1-a6a8-19d4962e8942 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:58.515426 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d2b0a-593b-4af1-a6a8-19d4962e8942 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:58.516270 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d2b0a-593b-4af1-a6a8-19d4962e8942 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:58.517398 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d2b0a-593b-4af1-a6a8-19d4962e8942 None admin] RBAC: Adding query filter params (role.id=f8e6c4a0973a483dbdf8131bf851686a, scope.project.id=88724aa044b04819920009d1b5332589, user.id=7b2310ed5e31441faf1790f8f3fe4dad) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:58.517737 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d2b0a-593b-4af1-a6a8-19d4962e8942 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:58.518888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60580533-8a29-4571-b37c-7e964b87db3d None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:58.519369 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d2b0a-593b-4af1-a6a8-19d4962e8942 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:58.529731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 127/250] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:25:58 2022] GET /identity/v3/role_assignments?user.id=7b2310ed5e31441faf1790f8f3fe4dad&scope.project.id=88724aa044b04819920009d1b5332589&role.id=f8e6c4a0973a483dbdf8131bf851686a => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:58.562043 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8d55f-bed3-4eff-8495-505feff8a8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:58.562706 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8d55f-bed3-4eff-8495-505feff8a8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:58.562990 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8d55f-bed3-4eff-8495-505feff8a8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:58.563264 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8d55f-bed3-4eff-8495-505feff8a8f0 None admin] PATH_INFO: `/v3/users/329048f276444e0da9cb3132aa915713` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:58.564497 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c8d55f-bed3-4eff-8495-505feff8a8f0 None admin] RBAC: Authorizing `identity:get_user(user_id=329048f276444e0da9cb3132aa915713)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:58.566050 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c8d55f-bed3-4eff-8495-505feff8a8f0 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:58.570190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 124/251] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:58 2022] GET /identity/v3/users/329048f276444e0da9cb3132aa915713 => generated 297 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:58.578993 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d2b0a-593b-4af1-a6a8-19d4962e8942 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:58.747158 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d2b0a-593b-4af1-a6a8-19d4962e8942 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:58.770379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:58.796991 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd90a2c-7a59-419b-82db-3d12a5e5bdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:58.797672 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd90a2c-7a59-419b-82db-3d12a5e5bdda None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:58.797960 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd90a2c-7a59-419b-82db-3d12a5e5bdda None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:58.798235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd90a2c-7a59-419b-82db-3d12a5e5bdda None admin] PATH_INFO: `/v3/projects/88724aa044b04819920009d1b5332589` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:58.799425 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd90a2c-7a59-419b-82db-3d12a5e5bdda None admin] RBAC: Authorizing `identity:get_project(project_id=88724aa044b04819920009d1b5332589)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:58.800878 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd90a2c-7a59-419b-82db-3d12a5e5bdda None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:58.803309 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 128/252] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:25:58 2022] GET /identity/v3/projects/88724aa044b04819920009d1b5332589 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:58.809324 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8d55f-bed3-4eff-8495-505feff8a8f0 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:58.972880 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8d55f-bed3-4eff-8495-505feff8a8f0 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:58.998211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:59.026270 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7344a-7564-492c-bbb2-7afd34ce59e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:59.026961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7344a-7564-492c-bbb2-7afd34ce59e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:59.027245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7344a-7564-492c-bbb2-7afd34ce59e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:59.027519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7344a-7564-492c-bbb2-7afd34ce59e1 None admin] PATH_INFO: `/v3/projects/88724aa044b04819920009d1b5332589/users/329048f276444e0da9cb3132aa915713/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:59.029595 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7344a-7564-492c-bbb2-7afd34ce59e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=88724aa044b04819920009d1b5332589, user_id=329048f276444e0da9cb3132aa915713, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:59.031063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7344a-7564-492c-bbb2-7afd34ce59e1 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:59.051211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 125/253] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Aug 4 04:25:58 2022] PUT /identity/v3/projects/88724aa044b04819920009d1b5332589/users/329048f276444e0da9cb3132aa915713/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:25:59.133021 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c8f40-c743-4c67-9042-84476b0067e7 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:59.133371 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c8f40-c743-4c67-9042-84476b0067e7 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:59.133591 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c8f40-c743-4c67-9042-84476b0067e7 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:59.135336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 129/254] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:25:59.141298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47719643-e858-4d54-a720-e8fa177f9928 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:59.141602 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47719643-e858-4d54-a720-e8fa177f9928 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:59.141871 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47719643-e858-4d54-a720-e8fa177f9928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:59.162857 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-47719643-e858-4d54-a720-e8fa177f9928 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:59.164043 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-47719643-e858-4d54-a720-e8fa177f9928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:59.255736 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 126/255] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:59.294939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c8f40-c743-4c67-9042-84476b0067e7 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:59.420017 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c8f40-c743-4c67-9042-84476b0067e7 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:59.458698 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a4395-4479-4039-8243-0b5115df0784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:59.459398 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a4395-4479-4039-8243-0b5115df0784 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:59.459674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a4395-4479-4039-8243-0b5115df0784 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:59.459939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a4395-4479-4039-8243-0b5115df0784 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:59.470807 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a4395-4479-4039-8243-0b5115df0784 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:25:59.476391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-5b2a4395-4479-4039-8243-0b5115df0784 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:25:59.476391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:25:59.476391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:25:59.476391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:25:59.476391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:25:59.476391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:25:59.476391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:25:59.476391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:25:59.476391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:25:59.476391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:25:59.476391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:25:59.476391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:25:59.476391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:25:59.476391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:25:59.476391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:25:59.476391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:25:59.478517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:25:59.478517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:25:59.478517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:25:59.478517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:25:59.478517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:25:59.478517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:25:59.478517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:25:59.478517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:25:59.478517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:25:59.478517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:25:59.478517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:25:59.478517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:25:59.478517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:25:59.478517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:25:59.478517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:25:59.478517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:25:59.480443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:25:59.480443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:25:59.480443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:25:59.480443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:25:59.480443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:25:59.480443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:25:59.480443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:25:59.480443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:25:59.480443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:25:59.480443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:25:59.480443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:25:59.480443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:25:59.480443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 130/256] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:25:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:25:59.486860 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47719643-e858-4d54-a720-e8fa177f9928 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:59.606255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7f5e2-649b-4e62-ac41-24109c0a356e None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:59.606587 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7f5e2-649b-4e62-ac41-24109c0a356e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:59.606772 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7f5e2-649b-4e62-ac41-24109c0a356e None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:59.608113 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 131/257] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:25:59.617456 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6753478-e385-4d4e-8048-f829e1fb2d6f None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:59.617456 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6753478-e385-4d4e-8048-f829e1fb2d6f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:59.617456 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6753478-e385-4d4e-8048-f829e1fb2d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:59.624152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47719643-e858-4d54-a720-e8fa177f9928 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:59.634133 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:59.651984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-a6753478-e385-4d4e-8048-f829e1fb2d6f None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:25:59.653466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a6753478-e385-4d4e-8048-f829e1fb2d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:25:59.677814 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c84f86f-b97b-4c70-9d12-5540b119193d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:59.678358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84f86f-b97b-4c70-9d12-5540b119193d None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:59.678635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84f86f-b97b-4c70-9d12-5540b119193d None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:59.678904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84f86f-b97b-4c70-9d12-5540b119193d None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:59.679628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c84f86f-b97b-4c70-9d12-5540b119193d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:25:59.679930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c84f86f-b97b-4c70-9d12-5540b119193d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:59.681310 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c84f86f-b97b-4c70-9d12-5540b119193d None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:59.690486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 127/258] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:25:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:25:59.696345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c84f86f-b97b-4c70-9d12-5540b119193d None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:59.720564 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:59.776311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 132/259] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:25:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:25:59.821156 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6753478-e385-4d4e-8048-f829e1fb2d6f None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:59.828479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c84f86f-b97b-4c70-9d12-5540b119193d None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:59.865611 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ccced-032c-499f-92d0-3081327e156f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:25:59.866322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ccced-032c-499f-92d0-3081327e156f None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:25:59.866599 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ccced-032c-499f-92d0-3081327e156f None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:25:59.866865 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ccced-032c-499f-92d0-3081327e156f None admin] PATH_INFO: `/v3/users/bd5f3d107ac94abcad21dfdd9b166ae7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:25:59.868052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ccced-032c-499f-92d0-3081327e156f None admin] RBAC: Authorizing `identity:get_user(user_id=bd5f3d107ac94abcad21dfdd9b166ae7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:25:59.869480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ccced-032c-499f-92d0-3081327e156f None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:25:59.871225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 128/260] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:25:59 2022] GET /identity/v3/users/bd5f3d107ac94abcad21dfdd9b166ae7 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:25:59.877179 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ccced-032c-499f-92d0-3081327e156f None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:25:59.948187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6753478-e385-4d4e-8048-f829e1fb2d6f None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:25:59.951797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:25:59.999357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b1bd2-0464-4f28-93b8-77a87f244091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:00.000040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b1bd2-0464-4f28-93b8-77a87f244091 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:00.000372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b1bd2-0464-4f28-93b8-77a87f244091 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:00.000654 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b1bd2-0464-4f28-93b8-77a87f244091 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:00.020614 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b1bd2-0464-4f28-93b8-77a87f244091 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:00.025836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-503b1bd2-0464-4f28-93b8-77a87f244091 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:26:00.025836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:00.025836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:00.025836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:00.025836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:00.025836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:00.025836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:00.025836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:00.025836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:00.025836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:00.025836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:00.025836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:00.025836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:00.025836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:00.025836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:00.025836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:00.027869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:00.027869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:00.027869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:00.027869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:00.027869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:00.027869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:00.027869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:00.027869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:00.027869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:00.027869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:00.027869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:00.027869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:00.027869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:00.027869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:00.027869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:00.027869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:00.029942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:00.029942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:00.029942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:00.029942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:00.029942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:00.029942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:00.029942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:00.029942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:00.029942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:00.029942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:00.029942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:26:00.029942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:00.037775 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 133/261] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Aug 4 04:25:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:00.037992 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ccced-032c-499f-92d0-3081327e156f None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:00.038597 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b1bd2-0464-4f28-93b8-77a87f244091 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:00.086703 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc19652-ea98-4ea2-a226-c5d9d59694fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:00.087346 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc19652-ea98-4ea2-a226-c5d9d59694fd None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:00.087632 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc19652-ea98-4ea2-a226-c5d9d59694fd None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:00.087905 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc19652-ea98-4ea2-a226-c5d9d59694fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:00.096994 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:00.181078 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc19652-ea98-4ea2-a226-c5d9d59694fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:00.182900 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc19652-ea98-4ea2-a226-c5d9d59694fd None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:00.233415 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b1bd2-0464-4f28-93b8-77a87f244091 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:00.267184 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:00.317570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5203821f-6ce2-4266-a0c5-8e8a4b710fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:00.317570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203821f-6ce2-4266-a0c5-8e8a4b710fbb None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:00.317570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203821f-6ce2-4266-a0c5-8e8a4b710fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:00.317570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203821f-6ce2-4266-a0c5-8e8a4b710fbb None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:00.317980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5203821f-6ce2-4266-a0c5-8e8a4b710fbb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:00.317980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5203821f-6ce2-4266-a0c5-8e8a4b710fbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:00.317980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5203821f-6ce2-4266-a0c5-8e8a4b710fbb None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:00.344019 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 134/262] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:00.350740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5203821f-6ce2-4266-a0c5-8e8a4b710fbb None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:00.374995 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 129/263] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:25:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:26:00.381128 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc19652-ea98-4ea2-a226-c5d9d59694fd None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:00.496944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:00.527263 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5203821f-6ce2-4266-a0c5-8e8a4b710fbb None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:00.583258 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ccb81-a80f-4678-b8f7-799479f73519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:00.584024 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ccb81-a80f-4678-b8f7-799479f73519 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:00.584310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ccb81-a80f-4678-b8f7-799479f73519 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:00.584584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ccb81-a80f-4678-b8f7-799479f73519 None admin] PATH_INFO: `/v3/users/bd5f3d107ac94abcad21dfdd9b166ae7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:00.585829 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ccb81-a80f-4678-b8f7-799479f73519 None admin] RBAC: Authorizing `identity:get_user(user_id=bd5f3d107ac94abcad21dfdd9b166ae7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:00.587536 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ccb81-a80f-4678-b8f7-799479f73519 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:00.589156 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 135/264] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:26:00 2022] GET /identity/v3/users/bd5f3d107ac94abcad21dfdd9b166ae7 => generated 283 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:00.591548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc19652-ea98-4ea2-a226-c5d9d59694fd None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:00.610976 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ccb81-a80f-4678-b8f7-799479f73519 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:00.669952 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18343ff-0670-49ff-b274-d48b6b80b8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:00.670648 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18343ff-0670-49ff-b274-d48b6b80b8da None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:00.670879 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18343ff-0670-49ff-b274-d48b6b80b8da None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:00.671204 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18343ff-0670-49ff-b274-d48b6b80b8da None admin] PATH_INFO: `/v3/projects/88724aa044b04819920009d1b5332589` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:00.675410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18343ff-0670-49ff-b274-d48b6b80b8da None admin] RBAC: Authorizing `identity:get_project(project_id=88724aa044b04819920009d1b5332589)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:00.677776 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18343ff-0670-49ff-b274-d48b6b80b8da None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:00.679867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 130/265] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:26:00 2022] GET /identity/v3/projects/88724aa044b04819920009d1b5332589 => generated 302 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:00.685826 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18343ff-0670-49ff-b274-d48b6b80b8da None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:00.698589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:00.811225 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ccb81-a80f-4678-b8f7-799479f73519 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:00.880837 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27825a0d-d657-4da4-b111-df97e2b61c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:00.881533 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27825a0d-d657-4da4-b111-df97e2b61c2a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:00.881839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27825a0d-d657-4da4-b111-df97e2b61c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:00.882263 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27825a0d-d657-4da4-b111-df97e2b61c2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:00.884539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18343ff-0670-49ff-b274-d48b6b80b8da None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:00.939988 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:00.941347 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3df143-6da5-4275-acd5-64e1d430f2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:00.942308 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3df143-6da5-4275-acd5-64e1d430f2aa None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:00.942789 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3df143-6da5-4275-acd5-64e1d430f2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:00.943258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3df143-6da5-4275-acd5-64e1d430f2aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:00.944264 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3df143-6da5-4275-acd5-64e1d430f2aa None admin] RBAC: Adding query filter params (role.id=4a6517e5890a49718303007dac0756c7, scope.project.id=88724aa044b04819920009d1b5332589, user.id=bd5f3d107ac94abcad21dfdd9b166ae7) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:00.944805 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3df143-6da5-4275-acd5-64e1d430f2aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:00.946556 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3df143-6da5-4275-acd5-64e1d430f2aa None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:00.963656 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 131/266] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:26:00 2022] GET /identity/v3/role_assignments?user.id=bd5f3d107ac94abcad21dfdd9b166ae7&scope.project.id=88724aa044b04819920009d1b5332589&role.id=4a6517e5890a49718303007dac0756c7 => generated 608 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:01.024939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27825a0d-d657-4da4-b111-df97e2b61c2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:01.026451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27825a0d-d657-4da4-b111-df97e2b61c2a None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:01.167289 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 136/267] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:26:01.173641 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3df143-6da5-4275-acd5-64e1d430f2aa None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:01.189494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d034984-6ff5-437a-92ad-db619fecae30 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:01.189832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d034984-6ff5-437a-92ad-db619fecae30 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:01.190089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d034984-6ff5-437a-92ad-db619fecae30 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:01.192087 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 137/268] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:01.197280 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ea5f9-7f74-4565-9ceb-0ed3b5f1e52e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:01.197599 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ea5f9-7f74-4565-9ceb-0ed3b5f1e52e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:01.197887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ea5f9-7f74-4565-9ceb-0ed3b5f1e52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:01.230061 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-e17ea5f9-7f74-4565-9ceb-0ed3b5f1e52e None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:01.231304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e17ea5f9-7f74-4565-9ceb-0ed3b5f1e52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:01.313595 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:01.345519 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:01.353297 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3df143-6da5-4275-acd5-64e1d430f2aa None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:01.360934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 138/269] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:26:01.391153 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ea5f9-7f74-4565-9ceb-0ed3b5f1e52e None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:01.394518 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf1025-fb53-4796-99ae-32e1a3d6bc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:01.396073 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf1025-fb53-4796-99ae-32e1a3d6bc57 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:01.396863 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf1025-fb53-4796-99ae-32e1a3d6bc57 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:01.397359 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf1025-fb53-4796-99ae-32e1a3d6bc57 None admin] PATH_INFO: `/v3/projects/88724aa044b04819920009d1b5332589` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:01.398839 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf1025-fb53-4796-99ae-32e1a3d6bc57 None admin] RBAC: Authorizing `identity:get_project(project_id=88724aa044b04819920009d1b5332589)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:01.400526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf1025-fb53-4796-99ae-32e1a3d6bc57 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:01.402909 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 132/270] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:26:01 2022] GET /identity/v3/projects/88724aa044b04819920009d1b5332589 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:01.434925 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf1025-fb53-4796-99ae-32e1a3d6bc57 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:01.510668 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ea5f9-7f74-4565-9ceb-0ed3b5f1e52e None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:01.548738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0c3c0-e2ea-4a52-a853-9555917ace68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:01.549788 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0c3c0-e2ea-4a52-a853-9555917ace68 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:01.550275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0c3c0-e2ea-4a52-a853-9555917ace68 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:01.550808 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0c3c0-e2ea-4a52-a853-9555917ace68 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:01.563528 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0c3c0-e2ea-4a52-a853-9555917ace68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:01.569573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-1cb0c3c0-e2ea-4a52-a853-9555917ace68 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:26:01.569573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:01.569573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:01.569573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:01.569573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:01.569573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:01.569573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:01.569573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:01.569573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:01.569573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:01.569573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:01.569573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:01.569573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:01.569573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:01.569573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:01.569573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:01.570439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:01.570439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:01.570439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:01.570439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:01.570439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:01.570439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:01.570439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:01.570439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:01.570439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:01.570439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:01.570439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:01.570439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:01.570439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:01.570439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:01.570439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:01.570439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:01.571238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:01.571238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:01.571238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:01.571238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:01.571238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:01.571238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:01.571238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:01.571238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:01.571238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:01.571238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:01.571238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:26:01.571238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:01.577506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 139/271] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:01.580510 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0c3c0-e2ea-4a52-a853-9555917ace68 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:01.640007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:01.679713 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf1025-fb53-4796-99ae-32e1a3d6bc57 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:01.696807 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0c3c0-e2ea-4a52-a853-9555917ace68 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:01.734718 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd37d295-3c54-4fea-9705-a522c3936000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:01.735709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37d295-3c54-4fea-9705-a522c3936000 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:01.736194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37d295-3c54-4fea-9705-a522c3936000 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:01.736666 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37d295-3c54-4fea-9705-a522c3936000 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:01.737765 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd37d295-3c54-4fea-9705-a522c3936000 None admin] RBAC: Adding query filter params (name=member) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:01.738272 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd37d295-3c54-4fea-9705-a522c3936000 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:01.740473 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd37d295-3c54-4fea-9705-a522c3936000 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:01.748150 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52e975-dbd3-4237-abbb-67c738a69c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:01.749048 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52e975-dbd3-4237-abbb-67c738a69c64 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:01.749504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52e975-dbd3-4237-abbb-67c738a69c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:01.749959 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52e975-dbd3-4237-abbb-67c738a69c64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:01.750903 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52e975-dbd3-4237-abbb-67c738a69c64 None admin] RBAC: Adding query filter params (role.id=7a855ec41b944458821d76509b481268, scope.project.id=88724aa044b04819920009d1b5332589, user.id=bd5f3d107ac94abcad21dfdd9b166ae7) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:01.751428 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52e975-dbd3-4237-abbb-67c738a69c64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:01.753068 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52e975-dbd3-4237-abbb-67c738a69c64 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:01.755451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 140/272] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:26:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:01.757156 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856aeb2-52bf-45b2-85ef-7395a64f8708 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:01.757524 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856aeb2-52bf-45b2-85ef-7395a64f8708 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:01.757840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856aeb2-52bf-45b2-85ef-7395a64f8708 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:01.759229 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 141/273] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:01.760607 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856aeb2-52bf-45b2-85ef-7395a64f8708 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:01.765688 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 133/274] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:26:01 2022] GET /identity/v3/role_assignments?user.id=bd5f3d107ac94abcad21dfdd9b166ae7&scope.project.id=88724aa044b04819920009d1b5332589&role.id=7a855ec41b944458821d76509b481268 => generated 608 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:01.767206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be7ded-f048-4fb0-94f7-df7f07cafeb5 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:01.767697 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be7ded-f048-4fb0-94f7-df7f07cafeb5 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:01.768172 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be7ded-f048-4fb0-94f7-df7f07cafeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:01.784760 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:01.810365 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-c1be7ded-f048-4fb0-94f7-df7f07cafeb5 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:01.811601 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c1be7ded-f048-4fb0-94f7-df7f07cafeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:01.934431 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856aeb2-52bf-45b2-85ef-7395a64f8708 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:01.951794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:01.963716 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 134/275] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:01.964834 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694c597-46da-49cb-9255-bfb0d801cda3 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:01.965125 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694c597-46da-49cb-9255-bfb0d801cda3 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:01.965399 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694c597-46da-49cb-9255-bfb0d801cda3 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:01.970693 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 135/276] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:01.976786 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c763a8-f09b-40d1-a1f4-d154d3dc9719 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:01.977410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c763a8-f09b-40d1-a1f4-d154d3dc9719 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:01.977903 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c763a8-f09b-40d1-a1f4-d154d3dc9719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:01.979559 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d454985-7932-43ed-bf85-d4ff34a02816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:01.980252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d454985-7932-43ed-bf85-d4ff34a02816 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:01.980544 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d454985-7932-43ed-bf85-d4ff34a02816 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:01.980823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d454985-7932-43ed-bf85-d4ff34a02816 None admin] PATH_INFO: `/v3/users/0e4ecbe2b9d94016a77a5fdfa0daf6a9` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:01.997513 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d454985-7932-43ed-bf85-d4ff34a02816 None admin] RBAC: Authorizing `identity:get_user(user_id=0e4ecbe2b9d94016a77a5fdfa0daf6a9)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:01.999322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d454985-7932-43ed-bf85-d4ff34a02816 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:01.999872 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-d8c763a8-f09b-40d1-a1f4-d154d3dc9719 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:02.001160 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d8c763a8-f09b-40d1-a1f4-d154d3dc9719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:02.007158 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 142/277] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:26:01 2022] GET /identity/v3/users/0e4ecbe2b9d94016a77a5fdfa0daf6a9 => generated 293 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:02.008102 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d454985-7932-43ed-bf85-d4ff34a02816 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:02.090157 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 136/278] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:02.091358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c763a8-f09b-40d1-a1f4-d154d3dc9719 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:02.125893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d454985-7932-43ed-bf85-d4ff34a02816 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:02.158948 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:02.162092 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3580ff-5374-4378-a905-c407079a798f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:02.162742 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3580ff-5374-4378-a905-c407079a798f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:02.163026 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3580ff-5374-4378-a905-c407079a798f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:02.163302 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3580ff-5374-4378-a905-c407079a798f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:02.174487 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3580ff-5374-4378-a905-c407079a798f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:02.180501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-6e3580ff-5374-4378-a905-c407079a798f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 04:26:02.180501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:02.180501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:02.180501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:02.180501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:02.180501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:02.180501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:02.180501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:02.180501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:02.180501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:02.180501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:02.180501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:02.180501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:02.180501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:02.180501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:02.180501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:02.181382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:02.181382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:02.181382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:02.181382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:02.181382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:02.181382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:02.181382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:02.181382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:02.181382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:02.181382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:02.181382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:02.181382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:02.181382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:02.181382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:02.181382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:02.181382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:02.182193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:02.182193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:02.182193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:02.182193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:02.182193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:02.182193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:02.182193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:02.182193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:02.182193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:02.182193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:02.182193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 04:26:02.182193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:02.182193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 143/279] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:02.182815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3580ff-5374-4378-a905-c407079a798f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:02.208399 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c763a8-f09b-40d1-a1f4-d154d3dc9719 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:02.216321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:02.246128 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24599a1-9b08-46bd-81f8-9a4eb2298d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:02.246749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24599a1-9b08-46bd-81f8-9a4eb2298d52 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:02.247030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24599a1-9b08-46bd-81f8-9a4eb2298d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:02.247301 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24599a1-9b08-46bd-81f8-9a4eb2298d52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:02.248091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24599a1-9b08-46bd-81f8-9a4eb2298d52 None admin] RBAC: Adding query filter params (role.id=e733372bae6047be956ad703f83a480e, user.id=0e4ecbe2b9d94016a77a5fdfa0daf6a9, scope.system=all) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:02.248398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24599a1-9b08-46bd-81f8-9a4eb2298d52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:02.249860 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24599a1-9b08-46bd-81f8-9a4eb2298d52 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:02.255220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 137/280] 10.222.0.47 () {56 vars in 1308 bytes} [Thu Aug 4 04:26:02 2022] GET /identity/v3/role_assignments?user.id=0e4ecbe2b9d94016a77a5fdfa0daf6a9&scope.system=all&role.id=e733372bae6047be956ad703f83a480e => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:02.256028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24599a1-9b08-46bd-81f8-9a4eb2298d52 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:02.293571 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3580ff-5374-4378-a905-c407079a798f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:02.338700 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cbb07-c68c-44ae-98e9-f4c6062a70a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:02.339363 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cbb07-c68c-44ae-98e9-f4c6062a70a0 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:02.339643 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cbb07-c68c-44ae-98e9-f4c6062a70a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:02.339937 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cbb07-c68c-44ae-98e9-f4c6062a70a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:02.341215 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558cbb07-c68c-44ae-98e9-f4c6062a70a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:02.342778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558cbb07-c68c-44ae-98e9-f4c6062a70a0 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:02.344659 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 144/281] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:02.345721 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a86227-fca3-42c9-9578-8e07b48ae7fa None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:02.346017 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a86227-fca3-42c9-9578-8e07b48ae7fa None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:02.346278 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a86227-fca3-42c9-9578-8e07b48ae7fa None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:02.347771 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 145/282] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:02.353236 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a86227-fca3-42c9-9578-8e07b48ae7fa None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:02.395938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24599a1-9b08-46bd-81f8-9a4eb2298d52 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:02.426921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:02.436824 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb1028-c0e7-4c91-8bb3-b783318a5a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:02.437449 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb1028-c0e7-4c91-8bb3-b783318a5a37 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:02.437775 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb1028-c0e7-4c91-8bb3-b783318a5a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:02.438046 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb1028-c0e7-4c91-8bb3-b783318a5a37 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:02.438777 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb1028-c0e7-4c91-8bb3-b783318a5a37 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:02.439079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb1028-c0e7-4c91-8bb3-b783318a5a37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:02.440442 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb1028-c0e7-4c91-8bb3-b783318a5a37 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:02.451027 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 138/283] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:26:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:02.452538 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6283a71-2d06-4d40-bedd-8adf078f6c0e None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:02.452831 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6283a71-2d06-4d40-bedd-8adf078f6c0e None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:02.453108 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6283a71-2d06-4d40-bedd-8adf078f6c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:02.471975 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-b6283a71-2d06-4d40-bedd-8adf078f6c0e None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:02.473064 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-b6283a71-2d06-4d40-bedd-8adf078f6c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:02.473922 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:02.478648 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a86227-fca3-42c9-9578-8e07b48ae7fa None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:02.522097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0de2e6-4b7d-4cb6-8fd7-b448e88ce7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:02.522717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0de2e6-4b7d-4cb6-8fd7-b448e88ce7fc None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:02.522993 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0de2e6-4b7d-4cb6-8fd7-b448e88ce7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:02.523258 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0de2e6-4b7d-4cb6-8fd7-b448e88ce7fc None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:02.523944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0de2e6-4b7d-4cb6-8fd7-b448e88ce7fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:02.525378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0de2e6-4b7d-4cb6-8fd7-b448e88ce7fc None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:02.549296 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 146/284] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Aug 4 04:26:02 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:26:02.549881 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 139/285] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:02.550169 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0de2e6-4b7d-4cb6-8fd7-b448e88ce7fc None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:02.577156 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6283a71-2d06-4d40-bedd-8adf078f6c0e None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:02.697182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0de2e6-4b7d-4cb6-8fd7-b448e88ce7fc None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:02.723761 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6283a71-2d06-4d40-bedd-8adf078f6c0e None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:02.742866 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fa0e6-9d94-4af3-a792-fb7704fa086b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:02.743525 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fa0e6-9d94-4af3-a792-fb7704fa086b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:02.743808 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fa0e6-9d94-4af3-a792-fb7704fa086b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:02.744083 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fa0e6-9d94-4af3-a792-fb7704fa086b None admin] PATH_INFO: `/v3/users/7b2310ed5e31441faf1790f8f3fe4dad` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:02.745276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6fa0e6-9d94-4af3-a792-fb7704fa086b None admin] RBAC: Authorizing `identity:get_user(user_id=7b2310ed5e31441faf1790f8f3fe4dad)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:02.746774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6fa0e6-9d94-4af3-a792-fb7704fa086b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:02.748526 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 147/286] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:26:02 2022] GET /identity/v3/users/7b2310ed5e31441faf1790f8f3fe4dad => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:02.748632 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:02.756234 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fa0e6-9d94-4af3-a792-fb7704fa086b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:02.767849 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df54896-99df-4496-8986-9cb2c85e9564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:02.768808 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df54896-99df-4496-8986-9cb2c85e9564 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:02.769292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df54896-99df-4496-8986-9cb2c85e9564 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:02.769712 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df54896-99df-4496-8986-9cb2c85e9564 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:02.784456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df54896-99df-4496-8986-9cb2c85e9564 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:02.788138 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:02.792795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-2df54896-99df-4496-8986-9cb2c85e9564 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:26:02.792795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:02.792795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:02.792795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:02.792795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:02.792795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:02.792795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:02.792795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:02.792795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:02.792795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:02.792795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:02.792795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:02.792795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:02.792795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:02.792795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:02.792795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:02.793785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:02.793785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:02.793785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:02.793785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:02.793785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:02.793785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:02.793785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:02.793785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:02.793785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:02.793785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:02.793785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:02.793785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:02.793785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:02.793785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:02.793785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:02.793785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:02.794539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:02.794539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:02.794539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:02.794539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:02.794539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:02.794539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:02.794539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:02.794539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:02.794539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:02.794539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:02.794539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:26:02.794539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:02.795158 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 140/287] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:26:02.801714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df54896-99df-4496-8986-9cb2c85e9564 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:02.890245 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fa0e6-9d94-4af3-a792-fb7704fa086b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:02.938806 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df54896-99df-4496-8986-9cb2c85e9564 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:02.946520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7698dd-9709-4ca5-bacc-eaa22bfbee1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:02.947711 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7698dd-9709-4ca5-bacc-eaa22bfbee1d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:02.947994 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7698dd-9709-4ca5-bacc-eaa22bfbee1d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:02.948269 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7698dd-9709-4ca5-bacc-eaa22bfbee1d None admin] PATH_INFO: `/v3/projects/88724aa044b04819920009d1b5332589` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:02.949931 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7698dd-9709-4ca5-bacc-eaa22bfbee1d None admin] RBAC: Authorizing `identity:get_project(project_id=88724aa044b04819920009d1b5332589)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:02.951736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7698dd-9709-4ca5-bacc-eaa22bfbee1d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:02.954216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 148/288] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:26:02 2022] GET /identity/v3/projects/88724aa044b04819920009d1b5332589 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:02.962771 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7698dd-9709-4ca5-bacc-eaa22bfbee1d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:02.984878 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b3116-edf8-4c8e-bd29-57563d0df7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:02.985422 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b3116-edf8-4c8e-bd29-57563d0df7d6 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:02.985730 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b3116-edf8-4c8e-bd29-57563d0df7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:02.986000 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b3116-edf8-4c8e-bd29-57563d0df7d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:02.986725 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b3116-edf8-4c8e-bd29-57563d0df7d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:02.987028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b3116-edf8-4c8e-bd29-57563d0df7d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:02.989735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b3116-edf8-4c8e-bd29-57563d0df7d6 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:02.999115 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 141/289] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:26:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:03.005087 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b3116-edf8-4c8e-bd29-57563d0df7d6 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:03.053114 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:03.076084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:03.090288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7698dd-9709-4ca5-bacc-eaa22bfbee1d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:03.115239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b3116-edf8-4c8e-bd29-57563d0df7d6 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:03.128428 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf2d6a-d5b6-4a67-9770-57e39e0c757c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:03.129214 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf2d6a-d5b6-4a67-9770-57e39e0c757c None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:03.130325 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf2d6a-d5b6-4a67-9770-57e39e0c757c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:03.130685 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf2d6a-d5b6-4a67-9770-57e39e0c757c None admin] PATH_INFO: `/v3/projects/88724aa044b04819920009d1b5332589/users/7b2310ed5e31441faf1790f8f3fe4dad/roles/f8e6c4a0973a483dbdf8131bf851686a` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:03.133032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf2d6a-d5b6-4a67-9770-57e39e0c757c None admin] RBAC: Authorizing `identity:create_grant(project_id=88724aa044b04819920009d1b5332589, user_id=7b2310ed5e31441faf1790f8f3fe4dad, role_id=f8e6c4a0973a483dbdf8131bf851686a)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:03.134871 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf2d6a-d5b6-4a67-9770-57e39e0c757c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:03.150034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 149/290] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Aug 4 04:26:02 2022] PUT /identity/v3/projects/88724aa044b04819920009d1b5332589/users/7b2310ed5e31441faf1790f8f3fe4dad/roles/f8e6c4a0973a483dbdf8131bf851686a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:26:03.153769 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3474a7c-a1e3-4d1a-8af6-2a2527a05a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:03.154422 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3474a7c-a1e3-4d1a-8af6-2a2527a05a02 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:03.154705 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3474a7c-a1e3-4d1a-8af6-2a2527a05a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:03.154977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3474a7c-a1e3-4d1a-8af6-2a2527a05a02 None admin] PATH_INFO: `/v3/users/329048f276444e0da9cb3132aa915713` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:03.156230 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3474a7c-a1e3-4d1a-8af6-2a2527a05a02 None admin] RBAC: Authorizing `identity:get_user(user_id=329048f276444e0da9cb3132aa915713)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:03.157737 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3474a7c-a1e3-4d1a-8af6-2a2527a05a02 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:03.159741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 142/291] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:26:03 2022] GET /identity/v3/users/329048f276444e0da9cb3132aa915713 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:03.165393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf2d6a-d5b6-4a67-9770-57e39e0c757c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:03.308754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf2d6a-d5b6-4a67-9770-57e39e0c757c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:03.362402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0c9132-81fa-40fa-b4a0-717e7791ff80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:03.363407 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c9132-81fa-40fa-b4a0-717e7791ff80 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:03.364522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c9132-81fa-40fa-b4a0-717e7791ff80 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:03.365660 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c9132-81fa-40fa-b4a0-717e7791ff80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:03.389906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:03.446926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0c9132-81fa-40fa-b4a0-717e7791ff80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:03.448901 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0c9132-81fa-40fa-b4a0-717e7791ff80 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:03.579494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 150/292] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 04:26:03.584783 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3474a7c-a1e3-4d1a-8af6-2a2527a05a02 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:03.721764 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3474a7c-a1e3-4d1a-8af6-2a2527a05a02 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:03.733496 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:03.766097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1f750-447d-41db-a576-79a2513f36c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:03.766736 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1f750-447d-41db-a576-79a2513f36c9 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:03.767011 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1f750-447d-41db-a576-79a2513f36c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:03.767278 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1f750-447d-41db-a576-79a2513f36c9 None admin] PATH_INFO: `/v3/projects/88724aa044b04819920009d1b5332589` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:03.768430 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1f750-447d-41db-a576-79a2513f36c9 None admin] RBAC: Authorizing `identity:get_project(project_id=88724aa044b04819920009d1b5332589)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:03.769894 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1f750-447d-41db-a576-79a2513f36c9 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:03.772145 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 143/293] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:26:03 2022] GET /identity/v3/projects/88724aa044b04819920009d1b5332589 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:03.778567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0c9132-81fa-40fa-b4a0-717e7791ff80 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:03.861344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:03.917401 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0c9132-81fa-40fa-b4a0-717e7791ff80 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:03.952395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492e5b2-4a70-4d86-9669-30fd4c148342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:03.953041 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492e5b2-4a70-4d86-9669-30fd4c148342 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:03.953289 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492e5b2-4a70-4d86-9669-30fd4c148342 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:03.953567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492e5b2-4a70-4d86-9669-30fd4c148342 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:03.954367 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492e5b2-4a70-4d86-9669-30fd4c148342 None admin] RBAC: Adding query filter params (role.id=e733372bae6047be956ad703f83a480e, scope.project.id=88724aa044b04819920009d1b5332589, user.id=329048f276444e0da9cb3132aa915713) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:03.954676 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492e5b2-4a70-4d86-9669-30fd4c148342 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:03.956118 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492e5b2-4a70-4d86-9669-30fd4c148342 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:03.963172 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 151/294] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:26:03 2022] GET /identity/v3/role_assignments?user.id=329048f276444e0da9cb3132aa915713&scope.project.id=88724aa044b04819920009d1b5332589&role.id=e733372bae6047be956ad703f83a480e => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:04.571885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173bcef-e091-4983-a6da-fe16151c0512 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:04.572261 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173bcef-e091-4983-a6da-fe16151c0512 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:04.572439 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173bcef-e091-4983-a6da-fe16151c0512 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:04.573910 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 144/295] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:04.579222 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a336a-529c-4049-a569-0bc92af91ded None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:04.579740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a336a-529c-4049-a569-0bc92af91ded None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:04.580226 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a336a-529c-4049-a569-0bc92af91ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:04.604884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b80a336a-529c-4049-a569-0bc92af91ded None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:04.606445 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b80a336a-529c-4049-a569-0bc92af91ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:04.704091 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 152/296] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:26:04.727731 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173bcef-e091-4983-a6da-fe16151c0512 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:04.836747 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173bcef-e091-4983-a6da-fe16151c0512 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:04.873733 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b972100-8580-4e0e-9552-9fc30812d7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:04.874733 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b972100-8580-4e0e-9552-9fc30812d7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:04.875209 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b972100-8580-4e0e-9552-9fc30812d7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:04.875678 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b972100-8580-4e0e-9552-9fc30812d7b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:04.886566 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b972100-8580-4e0e-9552-9fc30812d7b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:04.892192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-8b972100-8580-4e0e-9552-9fc30812d7b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:26:04.892192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:04.892192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:04.892192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:04.892192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:04.892192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:04.892192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:04.892192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:04.892192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:04.892192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:04.892192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:04.892192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:04.892192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:04.892192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:04.892192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:04.892192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:04.893076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:04.893076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:04.893076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:04.893076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:04.893076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:04.893076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:04.893076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:04.893076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:04.893076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:04.893076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:04.893076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:04.893076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:04.893076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:04.893076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:04.893076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:04.893076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:04.893888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:04.893888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:04.893888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:04.893888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:04.893888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:04.893888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:04.893888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:04.893888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:04.893888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:04.893888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:04.893888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:26:04.893888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:04.896825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 145/297] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:04.902785 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a336a-529c-4049-a569-0bc92af91ded None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:04.951139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:05.006283 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a336a-529c-4049-a569-0bc92af91ded None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:05.043129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0a7f0-5519-4310-82ea-b0216e166c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:05.043757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0a7f0-5519-4310-82ea-b0216e166c85 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:05.044038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0a7f0-5519-4310-82ea-b0216e166c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:05.044309 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0a7f0-5519-4310-82ea-b0216e166c85 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:05.045040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0a7f0-5519-4310-82ea-b0216e166c85 None admin] RBAC: Adding query filter params (name=member) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:05.045344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0a7f0-5519-4310-82ea-b0216e166c85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:05.047110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0a7f0-5519-4310-82ea-b0216e166c85 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:05.056647 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 153/298] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:26:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:05.062315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b972100-8580-4e0e-9552-9fc30812d7b5 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:05.103787 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:05.172125 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b972100-8580-4e0e-9552-9fc30812d7b5 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:05.208275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d3caa-7e52-43e7-8e54-d5b6116b6cdd None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:05.208790 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d3caa-7e52-43e7-8e54-d5b6116b6cdd None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:05.209034 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b252acd-fc55-4db3-a77a-76302f3ce508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:05.209186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d3caa-7e52-43e7-8e54-d5b6116b6cdd None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:05.209761 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b252acd-fc55-4db3-a77a-76302f3ce508 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:05.210042 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b252acd-fc55-4db3-a77a-76302f3ce508 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:05.210661 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b252acd-fc55-4db3-a77a-76302f3ce508 None admin] PATH_INFO: `/v3/users/0e4ecbe2b9d94016a77a5fdfa0daf6a9` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:05.210764 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 154/299] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:26:05.214314 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b252acd-fc55-4db3-a77a-76302f3ce508 None admin] RBAC: Authorizing `identity:get_user(user_id=0e4ecbe2b9d94016a77a5fdfa0daf6a9)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:05.215840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b252acd-fc55-4db3-a77a-76302f3ce508 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:05.216333 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab9afd-dd2b-42a5-af12-219899d81746 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:05.216720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab9afd-dd2b-42a5-af12-219899d81746 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:05.217090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab9afd-dd2b-42a5-af12-219899d81746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:05.221531 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 146/300] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:26:05 2022] GET /identity/v3/users/0e4ecbe2b9d94016a77a5fdfa0daf6a9 => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:05.223826 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b252acd-fc55-4db3-a77a-76302f3ce508 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:05.238054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-e0ab9afd-dd2b-42a5-af12-219899d81746 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:05.240283 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e0ab9afd-dd2b-42a5-af12-219899d81746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:05.321656 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 155/301] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:26:05.330899 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b252acd-fc55-4db3-a77a-76302f3ce508 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:05.334102 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:05.340403 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab9afd-dd2b-42a5-af12-219899d81746 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:05.369917 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ad086-b4c2-4402-909a-1a2f96262225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:05.370609 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ad086-b4c2-4402-909a-1a2f96262225 None admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:05.370804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ad086-b4c2-4402-909a-1a2f96262225 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:05.371053 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ad086-b4c2-4402-909a-1a2f96262225 None admin] PATH_INFO: `/v3/system/users/0e4ecbe2b9d94016a77a5fdfa0daf6a9/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:05.372504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137ad086-b4c2-4402-909a-1a2f96262225 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0e4ecbe2b9d94016a77a5fdfa0daf6a9, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:05.373907 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137ad086-b4c2-4402-909a-1a2f96262225 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:05.389101 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 147/302] 10.222.0.47 () {58 vars in 1334 bytes} [Thu Aug 4 04:26:05 2022] PUT /identity/v3/system/users/0e4ecbe2b9d94016a77a5fdfa0daf6a9/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:26:05.389101 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6538b-a212-4a71-865d-5ec1e744e626 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:05.389101 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6538b-a212-4a71-865d-5ec1e744e626 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:05.389101 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6538b-a212-4a71-865d-5ec1e744e626 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:05.389101 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 148/303] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:05.390216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:05.393498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3ca99-d3d4-4a45-8052-b0286a09e775 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:05.393819 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3ca99-d3d4-4a45-8052-b0286a09e775 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:05.394099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3ca99-d3d4-4a45-8052-b0286a09e775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:05.414793 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-c7f3ca99-d3d4-4a45-8052-b0286a09e775 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:05.416007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c7f3ca99-d3d4-4a45-8052-b0286a09e775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:05.469165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab9afd-dd2b-42a5-af12-219899d81746 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:05.506305 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 149/304] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:26:05.528530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e0663-fae5-4220-ac2a-a2f56d400d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:05.529127 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3ca99-d3d4-4a45-8052-b0286a09e775 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:05.529871 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e0663-fae5-4220-ac2a-a2f56d400d28 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:05.530369 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e0663-fae5-4220-ac2a-a2f56d400d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:05.530915 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e0663-fae5-4220-ac2a-a2f56d400d28 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:05.545022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700e0663-fae5-4220-ac2a-a2f56d400d28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:05.551913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-700e0663-fae5-4220-ac2a-a2f56d400d28 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 04:26:05.551913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:05.551913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:05.551913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:05.551913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:05.551913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:05.551913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:05.551913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:05.551913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:05.551913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:05.551913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:05.551913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:05.551913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:05.551913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:05.551913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:05.551913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:05.553048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:05.553048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:05.553048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:05.553048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:05.553048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:05.553048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:05.553048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:05.553048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:05.553048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:05.553048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:05.553048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:05.553048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:05.553048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:05.553048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:05.553048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:05.553048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:05.554322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:05.554322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:05.554322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:05.554322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:05.554322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:05.554322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:05.554322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:05.554322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:05.554322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:05.554322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:05.554322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 04:26:05.554322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:05.556898 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 156/305] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:26:05.563465 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e0663-fae5-4220-ac2a-a2f56d400d28 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:05.659420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:05.666443 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3ca99-d3d4-4a45-8052-b0286a09e775 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:05.679161 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:05.683325 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e0663-fae5-4220-ac2a-a2f56d400d28 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:05.707809 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc4a85-6074-4f3e-be4b-99431960b4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:05.708514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc4a85-6074-4f3e-be4b-99431960b4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:05.708746 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc4a85-6074-4f3e-be4b-99431960b4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:05.709014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc4a85-6074-4f3e-be4b-99431960b4f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:05.710249 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc4a85-6074-4f3e-be4b-99431960b4f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:05.711709 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc4a85-6074-4f3e-be4b-99431960b4f0 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:05.718662 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 150/306] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:05.718662 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f07a43-64dd-4bba-9f67-41c42bb18fa2 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:05.718662 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f07a43-64dd-4bba-9f67-41c42bb18fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:05.718662 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f07a43-64dd-4bba-9f67-41c42bb18fa2 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:05.718662 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 151/307] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:05.720809 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f07a43-64dd-4bba-9f67-41c42bb18fa2 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:05.729502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b14b9-551d-413f-909a-8ced82e1986b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:05.730562 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b14b9-551d-413f-909a-8ced82e1986b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:05.730855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b14b9-551d-413f-909a-8ced82e1986b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:05.731128 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b14b9-551d-413f-909a-8ced82e1986b None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:05.732228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b14b9-551d-413f-909a-8ced82e1986b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:05.732535 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b14b9-551d-413f-909a-8ced82e1986b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:05.734250 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b14b9-551d-413f-909a-8ced82e1986b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:05.748583 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 157/308] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:26:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:05.748583 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb58c2-c1cb-46cb-b3ee-1e7feb35fbe6 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:05.748583 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb58c2-c1cb-46cb-b3ee-1e7feb35fbe6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:05.748583 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb58c2-c1cb-46cb-b3ee-1e7feb35fbe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:05.764662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-73eb58c2-c1cb-46cb-b3ee-1e7feb35fbe6 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:05.765883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-73eb58c2-c1cb-46cb-b3ee-1e7feb35fbe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:05.836530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 158/309] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:05.837119 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f07a43-64dd-4bba-9f67-41c42bb18fa2 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:05.837575 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb58c2-c1cb-46cb-b3ee-1e7feb35fbe6 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:05.886848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8711110b-1b74-45fa-9db9-815a61588e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:05.887477 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711110b-1b74-45fa-9db9-815a61588e33 None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:05.887758 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711110b-1b74-45fa-9db9-815a61588e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:05.888032 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711110b-1b74-45fa-9db9-815a61588e33 None admin] PATH_INFO: `/v3/groups` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:05.888701 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8711110b-1b74-45fa-9db9-815a61588e33 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:05.890681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8711110b-1b74-45fa-9db9-815a61588e33 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:05.917781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 152/310] 10.222.0.47 () {60 vars in 1134 bytes} [Thu Aug 4 04:26:05 2022] POST /identity/v3/groups => generated 229 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:26:05.917781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8711110b-1b74-45fa-9db9-815a61588e33 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:05.950674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:05.962587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:05.964875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb58c2-c1cb-46cb-b3ee-1e7feb35fbe6 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:06.006522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4746cf-8af1-4d24-8878-aa41c3b8cdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:06.007173 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4746cf-8af1-4d24-8878-aa41c3b8cdbc None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:06.007446 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4746cf-8af1-4d24-8878-aa41c3b8cdbc None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:06.007722 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4746cf-8af1-4d24-8878-aa41c3b8cdbc None admin] PATH_INFO: `/v3/users/d7e2048ce00043a08552535e42fb54bc` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:06.024313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4746cf-8af1-4d24-8878-aa41c3b8cdbc None admin] RBAC: Authorizing `identity:get_user(user_id=d7e2048ce00043a08552535e42fb54bc)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:06.025913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4746cf-8af1-4d24-8878-aa41c3b8cdbc None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:06.027640 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 159/311] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:26:05 2022] GET /identity/v3/users/d7e2048ce00043a08552535e42fb54bc => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:06.033583 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4746cf-8af1-4d24-8878-aa41c3b8cdbc None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:06.073332 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8711110b-1b74-45fa-9db9-815a61588e33 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:06.120327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887185e-baac-4d4d-92c1-9ce4517684e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:06.121175 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887185e-baac-4d4d-92c1-9ce4517684e2 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:06.121456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887185e-baac-4d4d-92c1-9ce4517684e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:06.121757 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887185e-baac-4d4d-92c1-9ce4517684e2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:06.135196 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887185e-baac-4d4d-92c1-9ce4517684e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:06.141052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-3887185e-baac-4d4d-92c1-9ce4517684e2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 04:26:06.141052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:06.141052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:06.141052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:06.141052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:06.141052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:06.141052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:06.141052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:06.141052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:06.141052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:06.141052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:06.141052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:06.141052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:06.141052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:06.141052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:06.141052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:06.141965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:06.141965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:06.141965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:06.141965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:06.141965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:06.141965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:06.141965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:06.141965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:06.141965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:06.141965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:06.141965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:06.141965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:06.141965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:06.141965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:06.141965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:06.141965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:06.142732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:06.142732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:06.142732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:06.142732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:06.142732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:06.142732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:06.142732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:06.142732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:06.142732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:06.142732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:06.142732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 04:26:06.142732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:06.142732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 153/312] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:06.149950 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887185e-baac-4d4d-92c1-9ce4517684e2 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:06.159140 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4746cf-8af1-4d24-8878-aa41c3b8cdbc None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:06.212623 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d21178-5674-467a-a577-2e5eaa4f7d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:06.213154 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d21178-5674-467a-a577-2e5eaa4f7d8d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:06.213429 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d21178-5674-467a-a577-2e5eaa4f7d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:06.214224 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d21178-5674-467a-a577-2e5eaa4f7d8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:06.215039 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d21178-5674-467a-a577-2e5eaa4f7d8d None admin] RBAC: Adding query filter params (role.id=f8e6c4a0973a483dbdf8131bf851686a, user.id=d7e2048ce00043a08552535e42fb54bc, scope.system=all) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:06.215351 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d21178-5674-467a-a577-2e5eaa4f7d8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:06.217033 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d21178-5674-467a-a577-2e5eaa4f7d8d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:06.227177 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 160/313] 10.222.0.47 () {56 vars in 1308 bytes} [Thu Aug 4 04:26:06 2022] GET /identity/v3/role_assignments?user.id=d7e2048ce00043a08552535e42fb54bc&scope.system=all&role.id=f8e6c4a0973a483dbdf8131bf851686a => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:06.280009 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:06.283874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887185e-baac-4d4d-92c1-9ce4517684e2 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:06.324284 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c962ac0-de4b-4cf6-89f9-f436a67d7cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:06.325091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c962ac0-de4b-4cf6-89f9-f436a67d7cad None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:06.325562 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c962ac0-de4b-4cf6-89f9-f436a67d7cad None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:06.326029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c962ac0-de4b-4cf6-89f9-f436a67d7cad None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:06.326894 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c962ac0-de4b-4cf6-89f9-f436a67d7cad None admin] RBAC: Adding query filter params (name=reader) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:06.327314 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c962ac0-de4b-4cf6-89f9-f436a67d7cad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:06.329079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c962ac0-de4b-4cf6-89f9-f436a67d7cad None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:06.339208 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 154/314] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:26:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:06.344816 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d21178-5674-467a-a577-2e5eaa4f7d8d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:06.361439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:06.466454 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d21178-5674-467a-a577-2e5eaa4f7d8d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:06.508230 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85acd2eb-7201-4eb8-8ab4-039250a5bd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:06.508899 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acd2eb-7201-4eb8-8ab4-039250a5bd09 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:06.509182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acd2eb-7201-4eb8-8ab4-039250a5bd09 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:06.509454 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acd2eb-7201-4eb8-8ab4-039250a5bd09 None admin] PATH_INFO: `/v3/users/7b2310ed5e31441faf1790f8f3fe4dad` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:06.515919 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85acd2eb-7201-4eb8-8ab4-039250a5bd09 None admin] RBAC: Authorizing `identity:get_user(user_id=7b2310ed5e31441faf1790f8f3fe4dad)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:06.517382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85acd2eb-7201-4eb8-8ab4-039250a5bd09 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:06.519355 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 161/315] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:26:06 2022] GET /identity/v3/users/7b2310ed5e31441faf1790f8f3fe4dad => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:06.524877 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c962ac0-de4b-4cf6-89f9-f436a67d7cad None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:06.632359 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c962ac0-de4b-4cf6-89f9-f436a67d7cad None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:06.668117 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01b1ca-7197-445c-b7df-491104eabbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:06.668750 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01b1ca-7197-445c-b7df-491104eabbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:06.669035 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01b1ca-7197-445c-b7df-491104eabbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:06.669313 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01b1ca-7197-445c-b7df-491104eabbe1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:06.731246 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:06.740263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd01b1ca-7197-445c-b7df-491104eabbe1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:06.741798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd01b1ca-7197-445c-b7df-491104eabbe1 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:06.845081 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 155/316] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:26:06.851146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85acd2eb-7201-4eb8-8ab4-039250a5bd09 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:06.934790 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:06.955096 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85acd2eb-7201-4eb8-8ab4-039250a5bd09 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:06.990994 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338431fa-60c0-43ac-8ef6-bee5a8bdabf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:06.991562 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338431fa-60c0-43ac-8ef6-bee5a8bdabf2 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:06.991843 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338431fa-60c0-43ac-8ef6-bee5a8bdabf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:06.992115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338431fa-60c0-43ac-8ef6-bee5a8bdabf2 None admin] PATH_INFO: `/v3/projects/88724aa044b04819920009d1b5332589` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:06.993274 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338431fa-60c0-43ac-8ef6-bee5a8bdabf2 None admin] RBAC: Authorizing `identity:get_project(project_id=88724aa044b04819920009d1b5332589)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:06.994764 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338431fa-60c0-43ac-8ef6-bee5a8bdabf2 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:06.996715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 162/317] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:26:06 2022] GET /identity/v3/projects/88724aa044b04819920009d1b5332589 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:07.002516 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01b1ca-7197-445c-b7df-491104eabbe1 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:07.055083 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:07.109273 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01b1ca-7197-445c-b7df-491104eabbe1 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:07.150104 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950427d-74ba-4fe0-be82-208e4a17a9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:07.150725 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950427d-74ba-4fe0-be82-208e4a17a9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:07.151005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950427d-74ba-4fe0-be82-208e4a17a9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:07.151276 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950427d-74ba-4fe0-be82-208e4a17a9b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:07.152075 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0950427d-74ba-4fe0-be82-208e4a17a9b3 None admin] RBAC: Adding query filter params (role.id=f8e6c4a0973a483dbdf8131bf851686a, scope.project.id=88724aa044b04819920009d1b5332589, user.id=7b2310ed5e31441faf1790f8f3fe4dad) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:07.152383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0950427d-74ba-4fe0-be82-208e4a17a9b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:07.154226 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0950427d-74ba-4fe0-be82-208e4a17a9b3 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:07.161348 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 156/318] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:26:07 2022] GET /identity/v3/role_assignments?user.id=7b2310ed5e31441faf1790f8f3fe4dad&scope.project.id=88724aa044b04819920009d1b5332589&role.id=f8e6c4a0973a483dbdf8131bf851686a => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:07.480322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e49704-fc78-4b3f-89c5-2e483e669d22 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:07.480835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e49704-fc78-4b3f-89c5-2e483e669d22 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:07.480908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e49704-fc78-4b3f-89c5-2e483e669d22 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:07.482298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 163/319] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:07.487451 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bddce0-a7b3-435f-a59f-2fea17336d67 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:07.487745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bddce0-a7b3-435f-a59f-2fea17336d67 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:07.488018 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bddce0-a7b3-435f-a59f-2fea17336d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:07.507926 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-a6bddce0-a7b3-435f-a59f-2fea17336d67 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:07.509139 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-a6bddce0-a7b3-435f-a59f-2fea17336d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:07.576849 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 157/320] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:07.602234 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e49704-fc78-4b3f-89c5-2e483e669d22 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:07.708330 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e49704-fc78-4b3f-89c5-2e483e669d22 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:07.741860 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2c174-a3e0-4894-9175-dfb975fcd49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:07.742430 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2c174-a3e0-4894-9175-dfb975fcd49c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:07.742712 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2c174-a3e0-4894-9175-dfb975fcd49c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:07.742983 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2c174-a3e0-4894-9175-dfb975fcd49c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:07.752972 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2c174-a3e0-4894-9175-dfb975fcd49c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:07.758570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-1dc2c174-a3e0-4894-9175-dfb975fcd49c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:26:07.758570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:07.758570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:07.758570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:07.758570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:07.758570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:07.758570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:07.758570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:07.758570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:07.758570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:07.758570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:07.758570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:07.758570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:07.758570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:07.758570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:07.758570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:07.759419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:07.759419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:07.759419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:07.759419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:07.759419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:07.759419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:07.759419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:07.759419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:07.759419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:07.759419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:07.759419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:07.759419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:07.759419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:07.759419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:07.759419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:07.759419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:07.760227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:07.760227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:07.760227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:07.760227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:07.760227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:07.760227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:07.760227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:07.760227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:07.760227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:07.760227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:07.760227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:26:07.760227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:07.764738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 164/321] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:07.766155 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bddce0-a7b3-435f-a59f-2fea17336d67 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:07.809711 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:07.868589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bddce0-a7b3-435f-a59f-2fea17336d67 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:07.903174 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543dbb5-f066-4f80-91ca-915b2bd34d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:07.903788 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543dbb5-f066-4f80-91ca-915b2bd34d4e None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:07.904065 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543dbb5-f066-4f80-91ca-915b2bd34d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:07.904334 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543dbb5-f066-4f80-91ca-915b2bd34d4e None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:07.905055 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2543dbb5-f066-4f80-91ca-915b2bd34d4e None admin] RBAC: Adding query filter params (name=member) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:07.905355 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2543dbb5-f066-4f80-91ca-915b2bd34d4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:07.906733 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2543dbb5-f066-4f80-91ca-915b2bd34d4e None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:07.915444 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 158/322] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:26:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:07.920941 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2c174-a3e0-4894-9175-dfb975fcd49c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:07.932705 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:08.022904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2c174-a3e0-4894-9175-dfb975fcd49c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:08.058051 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78195c2-3f2e-4f42-9d33-e658272c735a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:08.058691 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78195c2-3f2e-4f42-9d33-e658272c735a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:08.058969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78195c2-3f2e-4f42-9d33-e658272c735a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:08.059239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78195c2-3f2e-4f42-9d33-e658272c735a None admin] PATH_INFO: `/v3/users/0e4ecbe2b9d94016a77a5fdfa0daf6a9` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:08.060392 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78195c2-3f2e-4f42-9d33-e658272c735a None admin] RBAC: Authorizing `identity:get_user(user_id=0e4ecbe2b9d94016a77a5fdfa0daf6a9)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:08.061867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78195c2-3f2e-4f42-9d33-e658272c735a None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:08.063606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 165/323] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:26:07 2022] GET /identity/v3/users/0e4ecbe2b9d94016a77a5fdfa0daf6a9 => generated 293 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:08.069474 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543dbb5-f066-4f80-91ca-915b2bd34d4e None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:08.164635 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5825e-111f-49ec-be88-3cf3556c9fa1 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:08.164954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5825e-111f-49ec-be88-3cf3556c9fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:08.165192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5825e-111f-49ec-be88-3cf3556c9fa1 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:08.166837 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 166/324] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:26:08.171868 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff4ead-ca05-4c8a-bd1a-24b62b92b60e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:08.172152 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff4ead-ca05-4c8a-bd1a-24b62b92b60e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:08.172481 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff4ead-ca05-4c8a-bd1a-24b62b92b60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:08.186401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543dbb5-f066-4f80-91ca-915b2bd34d4e None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:08.188716 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:08.190531 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-a4ff4ead-ca05-4c8a-bd1a-24b62b92b60e None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:08.192172 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a4ff4ead-ca05-4c8a-bd1a-24b62b92b60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:08.228174 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b501c87e-865f-404b-84bb-95b45556a8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:08.228801 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501c87e-865f-404b-84bb-95b45556a8bc None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:08.229100 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501c87e-865f-404b-84bb-95b45556a8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:08.229354 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501c87e-865f-404b-84bb-95b45556a8bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:08.230363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b501c87e-865f-404b-84bb-95b45556a8bc None admin] RBAC: Adding query filter params (role.id=e733372bae6047be956ad703f83a480e, user.id=0e4ecbe2b9d94016a77a5fdfa0daf6a9, scope.system=all) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:08.230674 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b501c87e-865f-404b-84bb-95b45556a8bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:08.232248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b501c87e-865f-404b-84bb-95b45556a8bc None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:08.249924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 159/325] 10.222.0.47 () {56 vars in 1308 bytes} [Thu Aug 4 04:26:08 2022] GET /identity/v3/role_assignments?user.id=0e4ecbe2b9d94016a77a5fdfa0daf6a9&scope.system=all&role.id=e733372bae6047be956ad703f83a480e => generated 510 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:08.264628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 167/326] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:26:08.288820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b501c87e-865f-404b-84bb-95b45556a8bc None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:08.377049 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7ccc3-6376-468e-878d-b677dd7e72f8 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:08.377356 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7ccc3-6376-468e-878d-b677dd7e72f8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:08.377642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7ccc3-6376-468e-878d-b677dd7e72f8 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:08.379035 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 168/327] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:26:08.385183 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df543194-5004-42d8-8424-92b79684b099 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:08.385470 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df543194-5004-42d8-8424-92b79684b099 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:08.385795 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df543194-5004-42d8-8424-92b79684b099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:08.400281 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:08.408580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-df543194-5004-42d8-8424-92b79684b099 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:08.411063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-df543194-5004-42d8-8424-92b79684b099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:08.418050 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b501c87e-865f-404b-84bb-95b45556a8bc None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:08.461794 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9615f64-0201-48a4-b8b9-1d88140157af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:08.462484 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9615f64-0201-48a4-b8b9-1d88140157af None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:08.463369 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9615f64-0201-48a4-b8b9-1d88140157af None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:08.463683 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9615f64-0201-48a4-b8b9-1d88140157af None admin] PATH_INFO: `/v3/domains/default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:08.464890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9615f64-0201-48a4-b8b9-1d88140157af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:08.467151 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9615f64-0201-48a4-b8b9-1d88140157af None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:08.468741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 160/328] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:08.475774 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9615f64-0201-48a4-b8b9-1d88140157af None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:08.490930 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 169/329] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:26:08.507829 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:08.513446 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df543194-5004-42d8-8424-92b79684b099 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:08.579696 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9615f64-0201-48a4-b8b9-1d88140157af None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:08.611060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df543194-5004-42d8-8424-92b79684b099 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:08.612586 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b2181b-37d2-4ae3-a224-ce54bc21f8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:08.613218 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2181b-37d2-4ae3-a224-ce54bc21f8cb None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:08.614367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2181b-37d2-4ae3-a224-ce54bc21f8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:08.614546 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2181b-37d2-4ae3-a224-ce54bc21f8cb None admin] PATH_INFO: `/v3/groups` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:08.615179 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b2181b-37d2-4ae3-a224-ce54bc21f8cb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:08.616594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b2181b-37d2-4ae3-a224-ce54bc21f8cb None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:08.632112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 161/330] 10.222.0.47 () {60 vars in 1134 bytes} [Thu Aug 4 04:26:08 2022] POST /identity/v3/groups => generated 226 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 04:26:08.647015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac67eed-1bad-4b85-bb4d-0f818be6fc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:08.647803 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac67eed-1bad-4b85-bb4d-0f818be6fc25 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:08.648146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac67eed-1bad-4b85-bb4d-0f818be6fc25 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:08.648482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac67eed-1bad-4b85-bb4d-0f818be6fc25 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:08.662352 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:08.673366 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac67eed-1bad-4b85-bb4d-0f818be6fc25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:08.679132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-7ac67eed-1bad-4b85-bb4d-0f818be6fc25 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 04:26:08.679132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:08.679132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:08.679132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:08.679132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:08.679132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:08.679132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:08.679132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:08.679132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:08.679132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:08.679132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:08.679132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:08.679132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:08.679132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:08.679132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:08.679132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:08.680135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:08.680135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:08.680135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:08.680135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:08.680135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:08.680135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:08.680135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:08.680135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:08.680135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:08.680135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:08.680135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:08.680135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:08.680135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:08.680135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:08.680135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:08.680135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:08.680908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:08.680908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:08.680908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:08.680908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:08.680908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:08.680908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:08.680908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:08.680908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:08.680908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:08.680908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:08.680908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 04:26:08.680908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:08.680908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 170/331] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:26:08.686743 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b2181b-37d2-4ae3-a224-ce54bc21f8cb None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:08.784207 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:08.795134 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b2181b-37d2-4ae3-a224-ce54bc21f8cb None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:08.830313 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea82c45-1d67-4f79-81ce-ecce26f4dbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:08.830869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea82c45-1d67-4f79-81ce-ecce26f4dbf7 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:08.831178 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea82c45-1d67-4f79-81ce-ecce26f4dbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:08.831477 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea82c45-1d67-4f79-81ce-ecce26f4dbf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:08.832772 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea82c45-1d67-4f79-81ce-ecce26f4dbf7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:08.833099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea82c45-1d67-4f79-81ce-ecce26f4dbf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:08.834479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea82c45-1d67-4f79-81ce-ecce26f4dbf7 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:08.842851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 162/332] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:26:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:08.849463 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac67eed-1bad-4b85-bb4d-0f818be6fc25 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:08.956592 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac67eed-1bad-4b85-bb4d-0f818be6fc25 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:08.993406 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ddaf6d-4668-447e-8cc6-39cb29695302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:08.994093 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ddaf6d-4668-447e-8cc6-39cb29695302 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:08.994372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ddaf6d-4668-447e-8cc6-39cb29695302 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:08.994646 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ddaf6d-4668-447e-8cc6-39cb29695302 None admin] PATH_INFO: `/v3/users/d7e2048ce00043a08552535e42fb54bc` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:08.995834 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ddaf6d-4668-447e-8cc6-39cb29695302 None admin] RBAC: Authorizing `identity:get_user(user_id=d7e2048ce00043a08552535e42fb54bc)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:08.997267 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ddaf6d-4668-447e-8cc6-39cb29695302 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:08.999405 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 171/333] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:26:08 2022] GET /identity/v3/users/d7e2048ce00043a08552535e42fb54bc => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:09.005884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea82c45-1d67-4f79-81ce-ecce26f4dbf7 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:09.115839 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea82c45-1d67-4f79-81ce-ecce26f4dbf7 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:09.153402 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76e7c0-8053-4970-91ad-a59e8abe6b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:09.154394 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76e7c0-8053-4970-91ad-a59e8abe6b44 None admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:09.154930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76e7c0-8053-4970-91ad-a59e8abe6b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:09.155226 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76e7c0-8053-4970-91ad-a59e8abe6b44 None admin] PATH_INFO: `/v3/system/users/d7e2048ce00043a08552535e42fb54bc/roles/f8e6c4a0973a483dbdf8131bf851686a` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:09.156694 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a76e7c0-8053-4970-91ad-a59e8abe6b44 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d7e2048ce00043a08552535e42fb54bc, role_id=f8e6c4a0973a483dbdf8131bf851686a)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:09.158102 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a76e7c0-8053-4970-91ad-a59e8abe6b44 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:09.167103 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 163/334] 10.222.0.47 () {58 vars in 1334 bytes} [Thu Aug 4 04:26:09 2022] PUT /identity/v3/system/users/d7e2048ce00043a08552535e42fb54bc/roles/f8e6c4a0973a483dbdf8131bf851686a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:26:10.933813 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5af1d-80aa-4172-92b6-70b34b981852 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:10.934322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5af1d-80aa-4172-92b6-70b34b981852 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:10.934396 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5af1d-80aa-4172-92b6-70b34b981852 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:10.935974 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 172/335] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:10.941266 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c52a70-19b3-4bf1-bdc0-88d90af7fcae None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:10.942509 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c52a70-19b3-4bf1-bdc0-88d90af7fcae None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:10.942804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c52a70-19b3-4bf1-bdc0-88d90af7fcae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:10.970675 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-54c52a70-19b3-4bf1-bdc0-88d90af7fcae None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:10.971921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-54c52a70-19b3-4bf1-bdc0-88d90af7fcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:10.991888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:11.067908 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 164/336] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:11.106834 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5af1d-80aa-4172-92b6-70b34b981852 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:11.137751 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ff7ba-183b-4e48-928c-fb5456f588f5 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:11.138069 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ff7ba-183b-4e48-928c-fb5456f588f5 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:11.138376 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ff7ba-183b-4e48-928c-fb5456f588f5 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:11.140930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 165/337] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:11.151329 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e41908-37a4-4c6f-8ff6-a899e171d228 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:11.151650 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e41908-37a4-4c6f-8ff6-a899e171d228 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:11.151960 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e41908-37a4-4c6f-8ff6-a899e171d228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:11.181745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-76e41908-37a4-4c6f-8ff6-a899e171d228 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:11.182926 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-76e41908-37a4-4c6f-8ff6-a899e171d228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:11.217403 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:11.299314 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5af1d-80aa-4172-92b6-70b34b981852 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:11.319056 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 166/338] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:26:11.346282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ce72f-2648-4677-b2d2-fa0cf15b2553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:11.346997 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ce72f-2648-4677-b2d2-fa0cf15b2553 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:11.347276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ce72f-2648-4677-b2d2-fa0cf15b2553 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:11.347546 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ce72f-2648-4677-b2d2-fa0cf15b2553 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:11.354437 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e41908-37a4-4c6f-8ff6-a899e171d228 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:11.372655 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ce72f-2648-4677-b2d2-fa0cf15b2553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:11.378427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-0a3ce72f-2648-4677-b2d2-fa0cf15b2553 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:26:11.378427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:11.378427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:11.378427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:11.378427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:11.378427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:11.378427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:11.378427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:11.378427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:11.378427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:11.378427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:11.378427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:11.378427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:11.378427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:11.378427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:11.378427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:11.380022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:11.380022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:11.380022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:11.380022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:11.380022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:11.380022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:11.380022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:11.380022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:11.380022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:11.380022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:11.380022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:11.380022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:11.380022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:11.380022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:11.380022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:11.380022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:11.382189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:11.382189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:11.382189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:11.382189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:11.382189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:11.382189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:11.382189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:11.382189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:11.382189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:11.382189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:11.382189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:26:11.382189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:11.382189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 173/339] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:11.403692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ce72f-2648-4677-b2d2-fa0cf15b2553 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:11.451697 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:11.475967 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e41908-37a4-4c6f-8ff6-a899e171d228 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:11.516603 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2984e55-213a-4904-b6c1-0f48cd8d4582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:11.517429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2984e55-213a-4904-b6c1-0f48cd8d4582 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:11.517963 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2984e55-213a-4904-b6c1-0f48cd8d4582 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:11.518450 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2984e55-213a-4904-b6c1-0f48cd8d4582 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:11.531122 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2984e55-213a-4904-b6c1-0f48cd8d4582 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:11.538318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-b2984e55-213a-4904-b6c1-0f48cd8d4582 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:26:11.538318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:11.538318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:11.538318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:11.538318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:11.538318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:11.538318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:11.538318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:11.538318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:11.538318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:11.538318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:11.538318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:11.538318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:11.538318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:11.538318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:11.538318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:11.539187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:11.539187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:11.539187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:11.539187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:11.539187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:11.539187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:11.539187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:11.539187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:11.539187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:11.539187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:11.539187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:11.539187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:11.539187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:11.539187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:11.539187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:11.539187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:11.539936 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:11.539936 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:11.539936 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:11.539936 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:11.539936 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:11.539936 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:11.539936 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:11.539936 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:11.539936 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:11.539936 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:11.539936 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:26:11.539936 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:11.542563 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 167/340] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Aug 4 04:26:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:26:11.548469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2984e55-213a-4904-b6c1-0f48cd8d4582 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:11.665978 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2984e55-213a-4904-b6c1-0f48cd8d4582 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:11.669623 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ce72f-2648-4677-b2d2-fa0cf15b2553 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:11.706959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:11.709992 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440fae3-5075-47cf-bcfb-12e1819252d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:11.711772 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440fae3-5075-47cf-bcfb-12e1819252d3 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:11.712245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440fae3-5075-47cf-bcfb-12e1819252d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:11.712703 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440fae3-5075-47cf-bcfb-12e1819252d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:11.713640 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9440fae3-5075-47cf-bcfb-12e1819252d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:11.714132 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9440fae3-5075-47cf-bcfb-12e1819252d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:11.715878 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9440fae3-5075-47cf-bcfb-12e1819252d3 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:11.725917 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:11.726823 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 168/341] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:11.729434 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07cd40-0526-498d-9721-672ea3e8bde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:11.730260 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07cd40-0526-498d-9721-672ea3e8bde2 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:11.730735 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07cd40-0526-498d-9721-672ea3e8bde2 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:11.731167 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07cd40-0526-498d-9721-672ea3e8bde2 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:11.732030 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07cd40-0526-498d-9721-672ea3e8bde2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:11.732483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07cd40-0526-498d-9721-672ea3e8bde2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:11.734412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440fae3-5075-47cf-bcfb-12e1819252d3 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:11.741409 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07cd40-0526-498d-9721-672ea3e8bde2 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:11.755230 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 174/342] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:26:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:11.761317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07cd40-0526-498d-9721-672ea3e8bde2 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:11.916968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440fae3-5075-47cf-bcfb-12e1819252d3 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:11.943496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07cd40-0526-498d-9721-672ea3e8bde2 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:11.960653 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76e428-49e4-4411-811a-71e1c97ac495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:11.962038 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76e428-49e4-4411-811a-71e1c97ac495 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:11.962503 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76e428-49e4-4411-811a-71e1c97ac495 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:11.963041 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76e428-49e4-4411-811a-71e1c97ac495 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:12.006396 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfbaae3-945c-4b21-95e4-fc5e366fbd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:12.006960 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbaae3-945c-4b21-95e4-fc5e366fbd8a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:12.007246 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbaae3-945c-4b21-95e4-fc5e366fbd8a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:12.007530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbaae3-945c-4b21-95e4-fc5e366fbd8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:12.042336 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76e428-49e4-4411-811a-71e1c97ac495 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:12.044275 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76e428-49e4-4411-811a-71e1c97ac495 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:12.058053 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:12.097284 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:12.145883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfbaae3-945c-4b21-95e4-fc5e366fbd8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:12.147431 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfbaae3-945c-4b21-95e4-fc5e366fbd8a None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:12.162148 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 169/343] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:26:12.171129 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76e428-49e4-4411-811a-71e1c97ac495 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:12.343017 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 175/344] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 04:26:12.345228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76322698-a0f2-48b4-8ee8-f98d7b15ebd1 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:12.345513 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76322698-a0f2-48b4-8ee8-f98d7b15ebd1 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:12.346555 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76322698-a0f2-48b4-8ee8-f98d7b15ebd1 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:12.347883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 176/345] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:12.348586 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76322698-a0f2-48b4-8ee8-f98d7b15ebd1 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:12.359019 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76e428-49e4-4411-811a-71e1c97ac495 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:12.365890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:12.430565 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e223b-cd61-47f0-891d-97bdc8d8215a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:12.431244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e223b-cd61-47f0-891d-97bdc8d8215a None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:12.431528 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e223b-cd61-47f0-891d-97bdc8d8215a None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:12.431980 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e223b-cd61-47f0-891d-97bdc8d8215a None admin] PATH_INFO: `/v3/projects/ba9c9c811ec54130a5b53525b32708d8` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:12.435303 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e223b-cd61-47f0-891d-97bdc8d8215a None admin] RBAC: Authorizing `identity:get_project(project_id=ba9c9c811ec54130a5b53525b32708d8)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:12.436272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e223b-cd61-47f0-891d-97bdc8d8215a None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:12.441889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 170/346] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:26:12 2022] GET /identity/v3/projects/ba9c9c811ec54130a5b53525b32708d8 => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:12.441889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795ab4f-ba48-455e-8e79-e5ab14353f1f None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:12.441889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795ab4f-ba48-455e-8e79-e5ab14353f1f None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:12.441889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795ab4f-ba48-455e-8e79-e5ab14353f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:12.467070 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-5795ab4f-ba48-455e-8e79-e5ab14353f1f None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:12.468276 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5795ab4f-ba48-455e-8e79-e5ab14353f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:12.509631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:12.541254 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76322698-a0f2-48b4-8ee8-f98d7b15ebd1 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:12.574336 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 171/347] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:12.575402 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b681118-4951-4a9f-9ffe-9e8eee0f417e None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:12.575693 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b681118-4951-4a9f-9ffe-9e8eee0f417e None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:12.575968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b681118-4951-4a9f-9ffe-9e8eee0f417e None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:12.578344 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 172/348] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:12.579097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b681118-4951-4a9f-9ffe-9e8eee0f417e None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:12.590326 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662d571-433d-44db-873f-982c48b793e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:12.591308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662d571-433d-44db-873f-982c48b793e6 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:12.592265 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662d571-433d-44db-873f-982c48b793e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:12.592730 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662d571-433d-44db-873f-982c48b793e6 None admin] PATH_INFO: `/v3/projects/ba9c9c811ec54130a5b53525b32708d8` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:12.594839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e662d571-433d-44db-873f-982c48b793e6 None admin] RBAC: Authorizing `identity:get_project(project_id=ba9c9c811ec54130a5b53525b32708d8)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:12.596731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e662d571-433d-44db-873f-982c48b793e6 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:12.600045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 177/349] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:26:12 2022] GET /identity/v3/projects/ba9c9c811ec54130a5b53525b32708d8 => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:12.601790 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da2d5f-5d0b-464c-ad16-d41b965cbc90 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:12.602720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da2d5f-5d0b-464c-ad16-d41b965cbc90 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:12.603191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da2d5f-5d0b-464c-ad16-d41b965cbc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:12.625335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-f2da2d5f-5d0b-464c-ad16-d41b965cbc90 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:12.626669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f2da2d5f-5d0b-464c-ad16-d41b965cbc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:12.708237 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:12.713228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 178/350] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:12.714773 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da2d5f-5d0b-464c-ad16-d41b965cbc90 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:12.716983 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b681118-4951-4a9f-9ffe-9e8eee0f417e None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:12.755685 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c2edc-9e5d-4fcf-bf27-9033e0938eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:12.756406 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c2edc-9e5d-4fcf-bf27-9033e0938eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:12.756687 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c2edc-9e5d-4fcf-bf27-9033e0938eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:12.756975 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c2edc-9e5d-4fcf-bf27-9033e0938eb5 None admin] PATH_INFO: `/v3/groups/01077af3c3244f3ab99a742d00bcf1b7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:12.763794 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c2edc-9e5d-4fcf-bf27-9033e0938eb5 None admin] RBAC: Authorizing `identity:get_group(group_id=01077af3c3244f3ab99a742d00bcf1b7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:12.765246 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c2edc-9e5d-4fcf-bf27-9033e0938eb5 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:12.767166 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 173/351] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Aug 4 04:26:12 2022] GET /identity/v3/groups/01077af3c3244f3ab99a742d00bcf1b7 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:12.767768 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c2edc-9e5d-4fcf-bf27-9033e0938eb5 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:12.824005 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:12.845322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da2d5f-5d0b-464c-ad16-d41b965cbc90 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:12.889491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c4bf9b-0e24-42c5-8d0a-a6b4c2dadf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:12.890332 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4bf9b-0e24-42c5-8d0a-a6b4c2dadf78 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:12.890617 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4bf9b-0e24-42c5-8d0a-a6b4c2dadf78 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:12.890945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4bf9b-0e24-42c5-8d0a-a6b4c2dadf78 None admin] PATH_INFO: `/v3/groups/01077af3c3244f3ab99a742d00bcf1b7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:12.892143 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c4bf9b-0e24-42c5-8d0a-a6b4c2dadf78 None admin] RBAC: Authorizing `identity:get_group(group_id=01077af3c3244f3ab99a742d00bcf1b7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:12.895784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c4bf9b-0e24-42c5-8d0a-a6b4c2dadf78 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:12.897760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 179/352] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Aug 4 04:26:12 2022] GET /identity/v3/groups/01077af3c3244f3ab99a742d00bcf1b7 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:12.898538 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c4bf9b-0e24-42c5-8d0a-a6b4c2dadf78 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:12.907753 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c2edc-9e5d-4fcf-bf27-9033e0938eb5 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:12.958147 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd50d32-4a09-4d3c-b275-3c0b4c7cb526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:12.959102 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd50d32-4a09-4d3c-b275-3c0b4c7cb526 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:12.959388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd50d32-4a09-4d3c-b275-3c0b4c7cb526 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:12.960169 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd50d32-4a09-4d3c-b275-3c0b4c7cb526 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:12.975176 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd50d32-4a09-4d3c-b275-3c0b4c7cb526 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:12.981577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-fbd50d32-4a09-4d3c-b275-3c0b4c7cb526 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 04:26:12.981577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:12.981577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:12.981577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:12.981577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:12.981577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:12.981577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:12.981577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:12.981577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:12.981577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:12.981577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:12.981577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:12.981577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:12.981577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:12.981577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:12.981577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:12.982514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:12.982514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:12.982514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:12.982514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:12.982514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:12.982514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:12.982514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:12.982514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:12.982514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:12.982514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:12.982514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:12.982514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:12.982514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:12.982514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:12.982514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:12.982514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:12.983345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:12.983345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:12.983345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:12.983345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:12.983345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:12.983345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:12.983345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:12.983345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:12.983345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:12.983345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:12.983345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 04:26:12.983345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:12.985625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 174/353] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:12.987625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd50d32-4a09-4d3c-b275-3c0b4c7cb526 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:13.013095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:13.037918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c4bf9b-0e24-42c5-8d0a-a6b4c2dadf78 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:13.080759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c290eeca-e32f-4c0a-8694-d3feaae2d8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:13.081139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290eeca-e32f-4c0a-8694-d3feaae2d8e8 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:13.081415 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290eeca-e32f-4c0a-8694-d3feaae2d8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:13.082442 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290eeca-e32f-4c0a-8694-d3feaae2d8e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:13.095881 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c290eeca-e32f-4c0a-8694-d3feaae2d8e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:13.101321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-c290eeca-e32f-4c0a-8694-d3feaae2d8e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:26:13.101321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:13.101321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:13.101321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:13.101321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:13.101321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:13.101321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:13.101321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:13.101321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:13.101321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:13.101321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:13.101321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:13.101321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:13.101321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:13.101321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:13.101321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:13.102299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:13.102299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:13.102299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:13.102299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:13.102299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:13.102299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:13.102299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:13.102299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:13.102299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:13.102299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:13.102299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:13.102299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:13.102299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:13.102299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:13.102299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:13.102299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:13.106419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:13.106419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:13.106419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:13.106419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:13.106419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:13.106419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:13.106419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:13.106419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:13.106419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:13.106419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:13.106419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:26:13.106419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:13.106419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 180/354] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:13.106419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c290eeca-e32f-4c0a-8694-d3feaae2d8e8 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:13.121884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:13.124515 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd50d32-4a09-4d3c-b275-3c0b4c7cb526 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:13.163630 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaeb7dc-8d95-494b-85c0-4fc192e5c6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:13.164423 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaeb7dc-8d95-494b-85c0-4fc192e5c6da None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:13.164529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaeb7dc-8d95-494b-85c0-4fc192e5c6da None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:13.164771 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaeb7dc-8d95-494b-85c0-4fc192e5c6da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:13.167089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaeb7dc-8d95-494b-85c0-4fc192e5c6da None admin] RBAC: Adding query filter params (group.id=01077af3c3244f3ab99a742d00bcf1b7, role.id=7a855ec41b944458821d76509b481268, scope.project.id=ba9c9c811ec54130a5b53525b32708d8) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:13.167089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaeb7dc-8d95-494b-85c0-4fc192e5c6da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:13.167308 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaeb7dc-8d95-494b-85c0-4fc192e5c6da None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:13.173948 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 175/355] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Aug 4 04:26:12 2022] GET /identity/v3/role_assignments?group.id=01077af3c3244f3ab99a742d00bcf1b7&scope.project.id=ba9c9c811ec54130a5b53525b32708d8&role.id=7a855ec41b944458821d76509b481268 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:13.175183 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca6b0b-cc1b-4e42-b247-02b5383bc2e7 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:13.175477 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca6b0b-cc1b-4e42-b247-02b5383bc2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:13.176379 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca6b0b-cc1b-4e42-b247-02b5383bc2e7 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:13.178134 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 176/356] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:13.178869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca6b0b-cc1b-4e42-b247-02b5383bc2e7 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:13.245854 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c290eeca-e32f-4c0a-8694-d3feaae2d8e8 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:13.296545 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7ccb0-6dda-4b5d-966d-b2fb0a21f904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:13.297239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7ccb0-6dda-4b5d-966d-b2fb0a21f904 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:13.297526 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7ccb0-6dda-4b5d-966d-b2fb0a21f904 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:13.297868 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7ccb0-6dda-4b5d-966d-b2fb0a21f904 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:13.298897 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7ccb0-6dda-4b5d-966d-b2fb0a21f904 None admin] RBAC: Adding query filter params (group.id=01077af3c3244f3ab99a742d00bcf1b7, role.id=e733372bae6047be956ad703f83a480e, scope.project.id=ba9c9c811ec54130a5b53525b32708d8) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:13.299176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7ccb0-6dda-4b5d-966d-b2fb0a21f904 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:13.300687 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7ccb0-6dda-4b5d-966d-b2fb0a21f904 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:13.311746 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:13.315735 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 181/357] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Aug 4 04:26:13 2022] GET /identity/v3/role_assignments?group.id=01077af3c3244f3ab99a742d00bcf1b7&scope.project.id=ba9c9c811ec54130a5b53525b32708d8&role.id=e733372bae6047be956ad703f83a480e => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:13.317710 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7ccb0-6dda-4b5d-966d-b2fb0a21f904 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:13.338042 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca6b0b-cc1b-4e42-b247-02b5383bc2e7 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:13.395796 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32d0f9-5a63-419e-88db-940f32bbcdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:13.396738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32d0f9-5a63-419e-88db-940f32bbcdf7 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:13.397200 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32d0f9-5a63-419e-88db-940f32bbcdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:13.397647 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32d0f9-5a63-419e-88db-940f32bbcdf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:13.398539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32d0f9-5a63-419e-88db-940f32bbcdf7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:13.399175 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32d0f9-5a63-419e-88db-940f32bbcdf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:13.401288 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32d0f9-5a63-419e-88db-940f32bbcdf7 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:13.412759 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 177/358] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:26:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:13.414445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba629d0d-984d-4fdd-8da0-2cdd5f5eac55 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:13.414940 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba629d0d-984d-4fdd-8da0-2cdd5f5eac55 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:13.415417 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba629d0d-984d-4fdd-8da0-2cdd5f5eac55 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:13.417177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 178/359] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:13.420577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bf331-23e7-4580-b396-1951d87e826b None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:13.421145 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bf331-23e7-4580-b396-1951d87e826b None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:13.421962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bf331-23e7-4580-b396-1951d87e826b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:13.444255 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-c49bf331-23e7-4580-b396-1951d87e826b None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:13.445956 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c49bf331-23e7-4580-b396-1951d87e826b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:13.459543 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:13.499796 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7ccb0-6dda-4b5d-966d-b2fb0a21f904 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:13.556343 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f70cec-7458-403c-8e3e-5637aeece3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:13.556982 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f70cec-7458-403c-8e3e-5637aeece3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:13.557292 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f70cec-7458-403c-8e3e-5637aeece3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:13.557590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f70cec-7458-403c-8e3e-5637aeece3a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:13.558442 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f70cec-7458-403c-8e3e-5637aeece3a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:13.558884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f70cec-7458-403c-8e3e-5637aeece3a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:13.561019 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f70cec-7458-403c-8e3e-5637aeece3a9 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:13.573396 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 182/360] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:26:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:13.574760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc473b44-b6ea-4f9e-9d54-51f8cba8370b None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:13.575188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc473b44-b6ea-4f9e-9d54-51f8cba8370b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:13.575444 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 179/361] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:13.575656 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc473b44-b6ea-4f9e-9d54-51f8cba8370b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:13.576625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49bf331-23e7-4580-b396-1951d87e826b None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:13.597499 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-cc473b44-b6ea-4f9e-9d54-51f8cba8370b None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:13.599281 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-cc473b44-b6ea-4f9e-9d54-51f8cba8370b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:13.674168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 183/362] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:13.674582 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:13.675115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc473b44-b6ea-4f9e-9d54-51f8cba8370b None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:13.705664 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49bf331-23e7-4580-b396-1951d87e826b None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:13.758287 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23dc6b-4073-42aa-b365-16a3af8d8a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:13.758668 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23dc6b-4073-42aa-b365-16a3af8d8a33 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:13.759009 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23dc6b-4073-42aa-b365-16a3af8d8a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:13.759284 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23dc6b-4073-42aa-b365-16a3af8d8a33 None admin] PATH_INFO: `/v3/users/d7e2048ce00043a08552535e42fb54bc` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:13.760806 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23dc6b-4073-42aa-b365-16a3af8d8a33 None admin] RBAC: Authorizing `identity:get_user(user_id=d7e2048ce00043a08552535e42fb54bc)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:13.762300 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23dc6b-4073-42aa-b365-16a3af8d8a33 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:13.764707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:13.765591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 180/363] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:26:13 2022] GET /identity/v3/users/d7e2048ce00043a08552535e42fb54bc => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:13.766337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23dc6b-4073-42aa-b365-16a3af8d8a33 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:13.805989 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc473b44-b6ea-4f9e-9d54-51f8cba8370b None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:13.863264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2204a-486b-4163-b35f-a651849c0fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:13.864265 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2204a-486b-4163-b35f-a651849c0fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:13.864558 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2204a-486b-4163-b35f-a651849c0fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:13.864836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2204a-486b-4163-b35f-a651849c0fc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:13.919842 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23dc6b-4073-42aa-b365-16a3af8d8a33 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:13.966331 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2204a-486b-4163-b35f-a651849c0fc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:13.967715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2204a-486b-4163-b35f-a651849c0fc8 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:13.969733 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc716aa-f9da-4dd8-9a91-399dffbc978e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:13.971399 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc716aa-f9da-4dd8-9a91-399dffbc978e None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:13.971870 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc716aa-f9da-4dd8-9a91-399dffbc978e None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:13.972327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc716aa-f9da-4dd8-9a91-399dffbc978e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:13.985243 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc716aa-f9da-4dd8-9a91-399dffbc978e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:13.993723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-7bc716aa-f9da-4dd8-9a91-399dffbc978e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:26:13.993723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:13.993723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:13.993723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:13.993723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:13.993723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:13.993723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:13.993723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:13.993723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:13.993723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:13.993723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:13.993723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:13.993723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:13.993723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:13.993723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:13.993723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:13.994682 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:13.994682 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:13.994682 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:13.994682 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:13.994682 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:13.994682 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:13.994682 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:13.994682 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:13.994682 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:13.994682 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:13.994682 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:13.994682 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:13.994682 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:13.994682 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:13.994682 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:13.994682 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:13.995534 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:13.995534 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:13.995534 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:13.995534 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:13.995534 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:13.995534 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:13.995534 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:13.995534 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:13.995534 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:13.995534 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:13.995534 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:26:13.995534 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:13.997865 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 181/364] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Aug 4 04:26:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:13.999045 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc716aa-f9da-4dd8-9a91-399dffbc978e None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:14.006644 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:14.067153 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:14.114707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 184/365] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:26:14.115641 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2204a-486b-4163-b35f-a651849c0fc8 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:14.157993 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc716aa-f9da-4dd8-9a91-399dffbc978e None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:14.210357 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c8953-897a-434e-9684-b3e20be268db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:14.211381 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c8953-897a-434e-9684-b3e20be268db None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:14.212125 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c8953-897a-434e-9684-b3e20be268db None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:14.212618 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c8953-897a-434e-9684-b3e20be268db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:14.230113 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c8953-897a-434e-9684-b3e20be268db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:14.234431 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2204a-486b-4163-b35f-a651849c0fc8 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:14.236569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-db3c8953-897a-434e-9684-b3e20be268db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:26:14.236569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:14.236569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:14.236569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:14.236569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:14.236569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:14.236569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:14.236569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:14.236569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:14.236569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:14.236569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:14.236569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:14.236569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:14.236569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:14.236569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:14.236569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:14.237446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:14.237446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:14.237446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:14.237446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:14.237446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:14.237446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:14.237446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:14.237446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:14.237446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:14.237446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:14.237446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:14.237446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:14.237446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:14.237446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:14.237446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:14.237446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:14.238800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:14.238800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:14.238800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:14.238800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:14.238800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:14.238800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:14.238800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:14.238800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:14.238800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:14.238800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:14.238800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:26:14.238800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:14.241427 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 182/366] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:26:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:14.243327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c8953-897a-434e-9684-b3e20be268db None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:14.286052 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71387d-555a-47e7-a551-75bb72d9a042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:14.286678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71387d-555a-47e7-a551-75bb72d9a042 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:14.286957 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71387d-555a-47e7-a551-75bb72d9a042 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:14.287226 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71387d-555a-47e7-a551-75bb72d9a042 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:14.288083 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb71387d-555a-47e7-a551-75bb72d9a042 None admin] RBAC: Adding query filter params (role.id=f8e6c4a0973a483dbdf8131bf851686a, user.id=d7e2048ce00043a08552535e42fb54bc, scope.system=all) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:14.288393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb71387d-555a-47e7-a551-75bb72d9a042 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:14.289840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb71387d-555a-47e7-a551-75bb72d9a042 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:14.303777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 185/367] 10.222.0.47 () {56 vars in 1308 bytes} [Thu Aug 4 04:26:14 2022] GET /identity/v3/role_assignments?user.id=d7e2048ce00043a08552535e42fb54bc&scope.system=all&role.id=f8e6c4a0973a483dbdf8131bf851686a => generated 510 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:14.304676 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71387d-555a-47e7-a551-75bb72d9a042 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:14.329176 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:14.339301 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:14.396199 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c8953-897a-434e-9684-b3e20be268db None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:14.440799 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71387d-555a-47e7-a551-75bb72d9a042 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:14.476555 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e330c-d0d6-4946-985e-655c3681b48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:14.477417 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e330c-d0d6-4946-985e-655c3681b48c None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:14.477535 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e330c-d0d6-4946-985e-655c3681b48c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:14.477807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e330c-d0d6-4946-985e-655c3681b48c None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:14.478563 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e330c-d0d6-4946-985e-655c3681b48c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:14.478853 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e330c-d0d6-4946-985e-655c3681b48c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:14.480387 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e330c-d0d6-4946-985e-655c3681b48c None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:14.483509 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d59d0-652e-4474-93fa-9dcc92035def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:14.484450 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d59d0-652e-4474-93fa-9dcc92035def None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:14.484943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d59d0-652e-4474-93fa-9dcc92035def None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:14.485413 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d59d0-652e-4474-93fa-9dcc92035def None admin] PATH_INFO: `/v3/projects/88724aa044b04819920009d1b5332589` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:14.487005 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d59d0-652e-4474-93fa-9dcc92035def None admin] RBAC: Authorizing `identity:get_project(project_id=88724aa044b04819920009d1b5332589)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:14.489373 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d59d0-652e-4474-93fa-9dcc92035def None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:14.492050 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 186/368] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:26:14 2022] GET /identity/v3/projects/88724aa044b04819920009d1b5332589 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:14.493090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d59d0-652e-4474-93fa-9dcc92035def None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:14.497811 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 183/369] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:14.498263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e330c-d0d6-4946-985e-655c3681b48c None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:14.614707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:14.622709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d59d0-652e-4474-93fa-9dcc92035def None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:14.674852 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94c773-63cf-490e-bff9-2f6fadc5e1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:14.676127 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94c773-63cf-490e-bff9-2f6fadc5e1b6 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:14.677183 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94c773-63cf-490e-bff9-2f6fadc5e1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:14.677961 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94c773-63cf-490e-bff9-2f6fadc5e1b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:14.678992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a94c773-63cf-490e-bff9-2f6fadc5e1b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:14.679515 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a94c773-63cf-490e-bff9-2f6fadc5e1b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:14.679627 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e330c-d0d6-4946-985e-655c3681b48c None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:14.681482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a94c773-63cf-490e-bff9-2f6fadc5e1b6 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:14.693236 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 187/370] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:14.695088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94c773-63cf-490e-bff9-2f6fadc5e1b6 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:14.719083 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:14.732599 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c45f5b-8013-4356-891a-6af569d1c818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:14.733173 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c45f5b-8013-4356-891a-6af569d1c818 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:14.733454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c45f5b-8013-4356-891a-6af569d1c818 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:14.733775 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c45f5b-8013-4356-891a-6af569d1c818 None admin] PATH_INFO: `/v3/groups/01077af3c3244f3ab99a742d00bcf1b7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:14.735065 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c45f5b-8013-4356-891a-6af569d1c818 None admin] RBAC: Authorizing `identity:get_group(group_id=01077af3c3244f3ab99a742d00bcf1b7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:14.736629 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c45f5b-8013-4356-891a-6af569d1c818 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:14.739582 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 184/371] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Aug 4 04:26:14 2022] GET /identity/v3/groups/01077af3c3244f3ab99a742d00bcf1b7 => generated 226 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:14.739887 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c45f5b-8013-4356-891a-6af569d1c818 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:14.847224 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94c773-63cf-490e-bff9-2f6fadc5e1b6 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:14.879585 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c45f5b-8013-4356-891a-6af569d1c818 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:14.887639 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f42e61-85f6-4a79-adaa-212d9c1b9b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:14.888387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f42e61-85f6-4a79-adaa-212d9c1b9b62 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:14.888673 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f42e61-85f6-4a79-adaa-212d9c1b9b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:14.888946 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f42e61-85f6-4a79-adaa-212d9c1b9b62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:14.924190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83564ca-e04c-4341-afc9-6c4211c5b8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:14.924806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:14.924922 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83564ca-e04c-4341-afc9-6c4211c5b8fa None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:14.925103 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83564ca-e04c-4341-afc9-6c4211c5b8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:14.925377 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83564ca-e04c-4341-afc9-6c4211c5b8fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:14.977739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f42e61-85f6-4a79-adaa-212d9c1b9b62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:14.979223 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f42e61-85f6-4a79-adaa-212d9c1b9b62 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:14.997687 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:15.007508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83564ca-e04c-4341-afc9-6c4211c5b8fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:15.009128 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83564ca-e04c-4341-afc9-6c4211c5b8fa None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:15.101963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 188/372] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:26:15.103366 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f42e61-85f6-4a79-adaa-212d9c1b9b62 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:15.148214 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 185/373] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:26:15.148937 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83564ca-e04c-4341-afc9-6c4211c5b8fa None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:15.183443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:15.240783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f42e61-85f6-4a79-adaa-212d9c1b9b62 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:15.278418 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:15.290186 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83564ca-e04c-4341-afc9-6c4211c5b8fa None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:15.295559 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c8be5-7492-4e26-bcbf-4e4a12d18a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:15.296678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c8be5-7492-4e26-bcbf-4e4a12d18a50 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:15.297145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c8be5-7492-4e26-bcbf-4e4a12d18a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:15.297585 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c8be5-7492-4e26-bcbf-4e4a12d18a50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:15.298531 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c8be5-7492-4e26-bcbf-4e4a12d18a50 None admin] RBAC: Adding query filter params (group.id=01077af3c3244f3ab99a742d00bcf1b7, role.id=e733372bae6047be956ad703f83a480e, scope.project.id=88724aa044b04819920009d1b5332589) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:15.298987 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c8be5-7492-4e26-bcbf-4e4a12d18a50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:15.300728 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c8be5-7492-4e26-bcbf-4e4a12d18a50 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:15.307870 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 189/374] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Aug 4 04:26:15 2022] GET /identity/v3/role_assignments?group.id=01077af3c3244f3ab99a742d00bcf1b7&scope.project.id=88724aa044b04819920009d1b5332589&role.id=e733372bae6047be956ad703f83a480e => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:15.308925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c8be5-7492-4e26-bcbf-4e4a12d18a50 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:15.337447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3ebd0-dbf4-496b-a25c-c0b5a4cdd573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:15.338405 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3ebd0-dbf4-496b-a25c-c0b5a4cdd573 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:15.338832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3ebd0-dbf4-496b-a25c-c0b5a4cdd573 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:15.339248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3ebd0-dbf4-496b-a25c-c0b5a4cdd573 None admin] PATH_INFO: `/v3/projects/88724aa044b04819920009d1b5332589` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:15.340622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3ebd0-dbf4-496b-a25c-c0b5a4cdd573 None admin] RBAC: Authorizing `identity:get_project(project_id=88724aa044b04819920009d1b5332589)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:15.342283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3ebd0-dbf4-496b-a25c-c0b5a4cdd573 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:15.344578 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 186/375] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:26:15 2022] GET /identity/v3/projects/88724aa044b04819920009d1b5332589 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:15.352530 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3ebd0-dbf4-496b-a25c-c0b5a4cdd573 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:15.436212 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c8be5-7492-4e26-bcbf-4e4a12d18a50 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:15.458757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:15.475934 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3ebd0-dbf4-496b-a25c-c0b5a4cdd573 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:15.476603 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de094b5f-c78e-4bda-9b4d-d3d6c07888ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:15.477275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de094b5f-c78e-4bda-9b4d-d3d6c07888ba None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:15.477587 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de094b5f-c78e-4bda-9b4d-d3d6c07888ba None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:15.477874 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de094b5f-c78e-4bda-9b4d-d3d6c07888ba None admin] PATH_INFO: `/v3/projects/312c7b7a94774ca2ba3533f4d13a2124` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:15.479131 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de094b5f-c78e-4bda-9b4d-d3d6c07888ba None admin] RBAC: Authorizing `identity:get_project(project_id=312c7b7a94774ca2ba3533f4d13a2124)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:15.481164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de094b5f-c78e-4bda-9b4d-d3d6c07888ba None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:15.483555 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 190/376] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:26:15 2022] GET /identity/v3/projects/312c7b7a94774ca2ba3533f4d13a2124 => generated 344 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:15.490155 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de094b5f-c78e-4bda-9b4d-d3d6c07888ba None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:15.522278 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d395a-28e3-4208-bab8-4330122f0a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:15.523222 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d395a-28e3-4208-bab8-4330122f0a6f None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:15.523661 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d395a-28e3-4208-bab8-4330122f0a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:15.524083 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d395a-28e3-4208-bab8-4330122f0a6f None admin] PATH_INFO: `/v3/groups/01077af3c3244f3ab99a742d00bcf1b7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:15.525579 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d395a-28e3-4208-bab8-4330122f0a6f None admin] RBAC: Authorizing `identity:get_group(group_id=01077af3c3244f3ab99a742d00bcf1b7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:15.527354 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d395a-28e3-4208-bab8-4330122f0a6f None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:15.533849 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 187/377] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Aug 4 04:26:15 2022] GET /identity/v3/groups/01077af3c3244f3ab99a742d00bcf1b7 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:15.537069 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d395a-28e3-4208-bab8-4330122f0a6f None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:15.577928 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:15.614132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de094b5f-c78e-4bda-9b4d-d3d6c07888ba None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:15.653368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c18c2-3b7e-426e-8e4f-113d305638b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:15.654104 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c18c2-3b7e-426e-8e4f-113d305638b8 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:15.654519 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c18c2-3b7e-426e-8e4f-113d305638b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:15.654871 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c18c2-3b7e-426e-8e4f-113d305638b8 None admin] PATH_INFO: `/v3/groups/d3122077a0cb44cea87c6ddef657da70` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:15.664856 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c18c2-3b7e-426e-8e4f-113d305638b8 None admin] RBAC: Authorizing `identity:get_group(group_id=d3122077a0cb44cea87c6ddef657da70)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:15.666614 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c18c2-3b7e-426e-8e4f-113d305638b8 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:15.668836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 191/378] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Aug 4 04:26:15 2022] GET /identity/v3/groups/d3122077a0cb44cea87c6ddef657da70 => generated 229 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:15.669285 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d395a-28e3-4208-bab8-4330122f0a6f None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:15.678631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c18c2-3b7e-426e-8e4f-113d305638b8 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:15.712342 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76a30e-e654-4f9a-8c78-34a4aa32a776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:15.712974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76a30e-e654-4f9a-8c78-34a4aa32a776 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:15.713298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76a30e-e654-4f9a-8c78-34a4aa32a776 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:15.713587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76a30e-e654-4f9a-8c78-34a4aa32a776 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:15.714388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76a30e-e654-4f9a-8c78-34a4aa32a776 None admin] RBAC: Adding query filter params (group.id=01077af3c3244f3ab99a742d00bcf1b7, role.id=7a855ec41b944458821d76509b481268, scope.project.id=88724aa044b04819920009d1b5332589) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:15.714696 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76a30e-e654-4f9a-8c78-34a4aa32a776 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:15.716361 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76a30e-e654-4f9a-8c78-34a4aa32a776 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:15.723125 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 188/379] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Aug 4 04:26:15 2022] GET /identity/v3/role_assignments?group.id=01077af3c3244f3ab99a742d00bcf1b7&scope.project.id=88724aa044b04819920009d1b5332589&role.id=7a855ec41b944458821d76509b481268 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:15.742560 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:15.801355 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c18c2-3b7e-426e-8e4f-113d305638b8 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:15.840417 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc1db5-2eb6-4058-9f12-487992b90929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:15.841144 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc1db5-2eb6-4058-9f12-487992b90929 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:15.841431 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc1db5-2eb6-4058-9f12-487992b90929 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:15.841737 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc1db5-2eb6-4058-9f12-487992b90929 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:15.842572 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc1db5-2eb6-4058-9f12-487992b90929 None admin] RBAC: Adding query filter params (group.id=d3122077a0cb44cea87c6ddef657da70, role.id=4a6517e5890a49718303007dac0756c7, scope.project.id=312c7b7a94774ca2ba3533f4d13a2124) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:15.842887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc1db5-2eb6-4058-9f12-487992b90929 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:15.844381 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc1db5-2eb6-4058-9f12-487992b90929 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:15.851812 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 192/380] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Aug 4 04:26:15 2022] GET /identity/v3/role_assignments?group.id=d3122077a0cb44cea87c6ddef657da70&scope.project.id=312c7b7a94774ca2ba3533f4d13a2124&role.id=4a6517e5890a49718303007dac0756c7 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:15.958558 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619afcd-b420-4261-8335-a2333636a43a None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:15.958558 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619afcd-b420-4261-8335-a2333636a43a None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:15.958558 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619afcd-b420-4261-8335-a2333636a43a None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:15.959436 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 189/381] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:15.965382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19a9e7-5ce2-492e-b927-0fb3d91def17 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:15.965701 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19a9e7-5ce2-492e-b927-0fb3d91def17 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:15.965969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19a9e7-5ce2-492e-b927-0fb3d91def17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:15.994793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-aa19a9e7-5ce2-492e-b927-0fb3d91def17 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:15.996023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-aa19a9e7-5ce2-492e-b927-0fb3d91def17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:16.084164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 193/382] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:16.108572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619afcd-b420-4261-8335-a2333636a43a None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:16.183767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0046c45-6a0d-4760-8310-bb6337f0b197 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:16.184287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0046c45-6a0d-4760-8310-bb6337f0b197 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:16.184287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0046c45-6a0d-4760-8310-bb6337f0b197 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:16.185677 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 194/383] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:26:16.191980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa9891-9d92-46fe-918b-9ffee87a5460 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:16.192336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa9891-9d92-46fe-918b-9ffee87a5460 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:16.192684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa9891-9d92-46fe-918b-9ffee87a5460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:16.236033 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-4dfa9891-9d92-46fe-918b-9ffee87a5460 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:16.240132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4dfa9891-9d92-46fe-918b-9ffee87a5460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:16.271775 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:16.289189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619afcd-b420-4261-8335-a2333636a43a None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:16.338712 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:16.346340 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e470ef9-e27a-4808-8411-ca382f75fcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:16.347421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e470ef9-e27a-4808-8411-ca382f75fcde None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:16.347892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e470ef9-e27a-4808-8411-ca382f75fcde None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:16.348396 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e470ef9-e27a-4808-8411-ca382f75fcde None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:16.378546 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e470ef9-e27a-4808-8411-ca382f75fcde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:16.386924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-5e470ef9-e27a-4808-8411-ca382f75fcde None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:26:16.386924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:16.386924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:16.386924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:16.386924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:16.386924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:16.386924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:16.386924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:16.386924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:16.386924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:16.386924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:16.386924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:16.386924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:16.386924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:16.386924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:16.386924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:16.387875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:16.387875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:16.387875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:16.387875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:16.387875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:16.387875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:16.387875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:16.387875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:16.387875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:16.387875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:16.387875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:16.387875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:16.387875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:16.387875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:16.387875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:16.387875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:16.388715 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:16.388715 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:16.388715 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:16.388715 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:16.388715 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:16.388715 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:16.388715 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:16.388715 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:16.388715 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:16.388715 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:16.388715 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:26:16.388715 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:16.388715 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 190/384] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Aug 4 04:26:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:16.395792 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 195/385] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:26:16.395984 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e470ef9-e27a-4808-8411-ca382f75fcde None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:16.427214 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa9891-9d92-46fe-918b-9ffee87a5460 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:16.531489 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e470ef9-e27a-4808-8411-ca382f75fcde None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:16.579708 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa9891-9d92-46fe-918b-9ffee87a5460 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:16.593654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de6e06-b254-4ce3-904a-2673d4e40bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:16.593654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de6e06-b254-4ce3-904a-2673d4e40bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:16.593654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de6e06-b254-4ce3-904a-2673d4e40bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:16.593654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de6e06-b254-4ce3-904a-2673d4e40bf4 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:16.594021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de6e06-b254-4ce3-904a-2673d4e40bf4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:16.594021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de6e06-b254-4ce3-904a-2673d4e40bf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:16.595700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de6e06-b254-4ce3-904a-2673d4e40bf4 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:16.610031 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 191/386] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:16.616972 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de6e06-b254-4ce3-904a-2673d4e40bf4 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:16.646219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:16.667531 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281d480-4668-4972-bc25-e9dc8ae453bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:16.668202 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281d480-4668-4972-bc25-e9dc8ae453bc None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:16.668483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281d480-4668-4972-bc25-e9dc8ae453bc None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:16.668755 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281d480-4668-4972-bc25-e9dc8ae453bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:16.697403 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b281d480-4668-4972-bc25-e9dc8ae453bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:16.700759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-b281d480-4668-4972-bc25-e9dc8ae453bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:26:16.700759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:16.700759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:16.700759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:16.700759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:16.700759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:16.700759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:16.700759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:16.700759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:16.700759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:16.700759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:16.700759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:16.700759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:16.700759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:16.700759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:16.700759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:16.701717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:16.701717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:16.701717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:16.701717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:16.701717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:16.701717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:16.701717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:16.701717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:16.701717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:16.701717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:16.701717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:16.701717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:16.701717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:16.701717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:16.701717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:16.701717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:16.702506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:16.702506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:16.702506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:16.702506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:16.702506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:16.702506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:16.702506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:16.702506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:16.702506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:16.702506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:16.702506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:26:16.702506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:16.710655 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 196/387] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:16.712028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281d480-4668-4972-bc25-e9dc8ae453bc None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:16.757688 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de6e06-b254-4ce3-904a-2673d4e40bf4 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:16.798829 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:16.811400 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e66c4-0772-4956-b672-72192108a1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:16.812113 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e66c4-0772-4956-b672-72192108a1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:16.812403 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e66c4-0772-4956-b672-72192108a1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:16.812678 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e66c4-0772-4956-b672-72192108a1f2 None admin] PATH_INFO: `/v3/groups/01077af3c3244f3ab99a742d00bcf1b7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:16.813905 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e66c4-0772-4956-b672-72192108a1f2 None admin] RBAC: Authorizing `identity:get_group(group_id=01077af3c3244f3ab99a742d00bcf1b7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:16.815347 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e66c4-0772-4956-b672-72192108a1f2 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:16.819638 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 192/388] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Aug 4 04:26:16 2022] GET /identity/v3/groups/01077af3c3244f3ab99a742d00bcf1b7 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:16.823160 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e66c4-0772-4956-b672-72192108a1f2 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:16.938924 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281d480-4668-4972-bc25-e9dc8ae453bc None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:17.001967 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:17.008805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bfddd8-bf21-4d91-9d11-274afa1aec99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:17.009572 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfddd8-bf21-4d91-9d11-274afa1aec99 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:17.009875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfddd8-bf21-4d91-9d11-274afa1aec99 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:17.010146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfddd8-bf21-4d91-9d11-274afa1aec99 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:17.010891 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bfddd8-bf21-4d91-9d11-274afa1aec99 None admin] RBAC: Adding query filter params (name=member) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:17.011202 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bfddd8-bf21-4d91-9d11-274afa1aec99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:17.012594 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bfddd8-bf21-4d91-9d11-274afa1aec99 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:17.027935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e66c4-0772-4956-b672-72192108a1f2 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:17.035104 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 197/389] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:26:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:17.041147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bfddd8-bf21-4d91-9d11-274afa1aec99 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:17.095873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c922c1f-d291-483a-ba0d-eb33f9c476d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:17.096624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c922c1f-d291-483a-ba0d-eb33f9c476d9 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:17.096911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c922c1f-d291-483a-ba0d-eb33f9c476d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:17.097183 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c922c1f-d291-483a-ba0d-eb33f9c476d9 None admin] PATH_INFO: `/v3/projects/ba9c9c811ec54130a5b53525b32708d8` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:17.099670 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c922c1f-d291-483a-ba0d-eb33f9c476d9 None admin] RBAC: Authorizing `identity:get_project(project_id=ba9c9c811ec54130a5b53525b32708d8)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:17.101784 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c922c1f-d291-483a-ba0d-eb33f9c476d9 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:17.103763 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 193/390] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:26:16 2022] GET /identity/v3/projects/ba9c9c811ec54130a5b53525b32708d8 => generated 298 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:17.111563 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c922c1f-d291-483a-ba0d-eb33f9c476d9 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:17.224305 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:17.231932 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bfddd8-bf21-4d91-9d11-274afa1aec99 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:17.275309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c922c1f-d291-483a-ba0d-eb33f9c476d9 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:17.318744 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4597f48-40cd-440d-ba61-8e606e2c158b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:17.319437 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4597f48-40cd-440d-ba61-8e606e2c158b None admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:17.319712 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4597f48-40cd-440d-ba61-8e606e2c158b None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:17.319984 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4597f48-40cd-440d-ba61-8e606e2c158b None admin] PATH_INFO: `/v3/projects/ba9c9c811ec54130a5b53525b32708d8/groups/01077af3c3244f3ab99a742d00bcf1b7/roles/7a855ec41b944458821d76509b481268` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:17.324388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4597f48-40cd-440d-ba61-8e606e2c158b None admin] RBAC: Authorizing `identity:create_grant(project_id=ba9c9c811ec54130a5b53525b32708d8, group_id=01077af3c3244f3ab99a742d00bcf1b7, role_id=7a855ec41b944458821d76509b481268)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:17.325956 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4597f48-40cd-440d-ba61-8e606e2c158b None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:17.327288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c7e82-99b9-4916-adc7-961428e35267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:17.327955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c7e82-99b9-4916-adc7-961428e35267 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:17.328238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c7e82-99b9-4916-adc7-961428e35267 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:17.328511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c7e82-99b9-4916-adc7-961428e35267 None admin] PATH_INFO: `/v3/groups/01077af3c3244f3ab99a742d00bcf1b7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:17.330996 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c7e82-99b9-4916-adc7-961428e35267 None admin] RBAC: Authorizing `identity:get_group(group_id=01077af3c3244f3ab99a742d00bcf1b7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:17.333266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c7e82-99b9-4916-adc7-961428e35267 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:17.338615 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 198/391] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Aug 4 04:26:17 2022] GET /identity/v3/groups/01077af3c3244f3ab99a742d00bcf1b7 => generated 226 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:17.344425 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c7e82-99b9-4916-adc7-961428e35267 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:17.347574 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 194/392] 10.222.0.47 () {58 vars in 1442 bytes} [Thu Aug 4 04:26:17 2022] PUT /identity/v3/projects/ba9c9c811ec54130a5b53525b32708d8/groups/01077af3c3244f3ab99a742d00bcf1b7/roles/7a855ec41b944458821d76509b481268 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:26:17.504330 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c7e82-99b9-4916-adc7-961428e35267 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:17.557933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6a494-b4bd-4ac5-a5dd-dceed4a773b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:17.558669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6a494-b4bd-4ac5-a5dd-dceed4a773b1 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:17.558955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6a494-b4bd-4ac5-a5dd-dceed4a773b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:17.559225 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6a494-b4bd-4ac5-a5dd-dceed4a773b1 None admin] PATH_INFO: `/v3/projects/ba9c9c811ec54130a5b53525b32708d8` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:17.560429 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6a494-b4bd-4ac5-a5dd-dceed4a773b1 None admin] RBAC: Authorizing `identity:get_project(project_id=ba9c9c811ec54130a5b53525b32708d8)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:17.561910 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6a494-b4bd-4ac5-a5dd-dceed4a773b1 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:17.563905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 199/393] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:26:17 2022] GET /identity/v3/projects/ba9c9c811ec54130a5b53525b32708d8 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:17.570409 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4597f48-40cd-440d-ba61-8e606e2c158b None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:17.593049 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:17.712440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4597f48-40cd-440d-ba61-8e606e2c158b None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:17.758492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5977ce4-80b4-44a7-8c15-b52f2410232e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:17.759149 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5977ce4-80b4-44a7-8c15-b52f2410232e None admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:17.759525 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5977ce4-80b4-44a7-8c15-b52f2410232e None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:17.759801 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5977ce4-80b4-44a7-8c15-b52f2410232e None admin] PATH_INFO: `/v3/projects/ba9c9c811ec54130a5b53525b32708d8/groups/01077af3c3244f3ab99a742d00bcf1b7/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:17.761896 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5977ce4-80b4-44a7-8c15-b52f2410232e None admin] RBAC: Authorizing `identity:create_grant(project_id=ba9c9c811ec54130a5b53525b32708d8, group_id=01077af3c3244f3ab99a742d00bcf1b7, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:17.763343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5977ce4-80b4-44a7-8c15-b52f2410232e None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:17.782967 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 195/394] 10.222.0.47 () {58 vars in 1442 bytes} [Thu Aug 4 04:26:17 2022] PUT /identity/v3/projects/ba9c9c811ec54130a5b53525b32708d8/groups/01077af3c3244f3ab99a742d00bcf1b7/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:26:18.136842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0386e5-c283-4253-ab66-7e4b74a48b57 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:18.137223 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0386e5-c283-4253-ab66-7e4b74a48b57 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:18.137400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0386e5-c283-4253-ab66-7e4b74a48b57 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:18.138807 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 200/395] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:18.144631 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341e90c-b393-4940-8c44-9b6fe8c75664 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:18.145169 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341e90c-b393-4940-8c44-9b6fe8c75664 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:18.145169 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341e90c-b393-4940-8c44-9b6fe8c75664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:18.171462 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-7341e90c-b393-4940-8c44-9b6fe8c75664 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:18.172678 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-7341e90c-b393-4940-8c44-9b6fe8c75664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:18.262243 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 196/396] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:18.332764 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0386e5-c283-4253-ab66-7e4b74a48b57 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:18.406336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:18.502349 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0386e5-c283-4253-ab66-7e4b74a48b57 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:18.539610 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74210d-5d9c-4313-9dee-3c1287f7a5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:18.540267 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74210d-5d9c-4313-9dee-3c1287f7a5fd None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:18.544435 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74210d-5d9c-4313-9dee-3c1287f7a5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:18.544714 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74210d-5d9c-4313-9dee-3c1287f7a5fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:18.556628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74210d-5d9c-4313-9dee-3c1287f7a5fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:18.562192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-cd74210d-5d9c-4313-9dee-3c1287f7a5fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:26:18.562192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:18.562192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:18.562192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:18.562192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:18.562192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:18.562192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:18.562192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:18.562192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:18.562192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:18.562192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:18.562192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:18.562192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:18.562192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:18.562192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:18.562192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:18.563076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:18.563076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:18.563076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:18.563076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:18.563076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:18.563076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:18.563076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:18.563076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:18.563076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:18.563076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:18.563076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:18.563076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:18.563076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:18.563076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:18.563076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:18.563076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:18.563859 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:18.563859 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:18.563859 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:18.563859 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:18.563859 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:18.563859 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:18.563859 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:18.563859 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:18.563859 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:18.563859 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:18.563859 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:26:18.563859 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:18.563859 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 201/397] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:18.569666 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341e90c-b393-4940-8c44-9b6fe8c75664 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:18.582588 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:18.617449 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b1825-f98c-4620-a476-cf423e9580ec None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:18.617993 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b1825-f98c-4620-a476-cf423e9580ec None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:18.618267 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b1825-f98c-4620-a476-cf423e9580ec None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:18.619865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 202/398] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:18.625352 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367e856-91be-4860-8126-6acf15fb2ec5 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:18.625660 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367e856-91be-4860-8126-6acf15fb2ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:18.625934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367e856-91be-4860-8126-6acf15fb2ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:18.645005 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-1367e856-91be-4860-8126-6acf15fb2ec5 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:18.646427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1367e856-91be-4860-8126-6acf15fb2ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:18.758139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 203/399] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:26:18.769797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341e90c-b393-4940-8c44-9b6fe8c75664 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:18.787578 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367e856-91be-4860-8126-6acf15fb2ec5 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:18.829395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:18.868283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800afda-bcf6-4518-b8ab-8b16ab09da2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:18.868986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800afda-bcf6-4518-b8ab-8b16ab09da2c None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:18.869279 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800afda-bcf6-4518-b8ab-8b16ab09da2c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:18.869575 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800afda-bcf6-4518-b8ab-8b16ab09da2c None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:18.870318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7800afda-bcf6-4518-b8ab-8b16ab09da2c None admin] RBAC: Adding query filter params (name=member) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:18.870949 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7800afda-bcf6-4518-b8ab-8b16ab09da2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:18.872053 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7800afda-bcf6-4518-b8ab-8b16ab09da2c None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:18.893196 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 197/400] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:26:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:18.901041 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800afda-bcf6-4518-b8ab-8b16ab09da2c None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:18.923774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367e856-91be-4860-8126-6acf15fb2ec5 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:18.965218 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01a390-dc36-423a-9b2c-0353f77e2b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:18.966236 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01a390-dc36-423a-9b2c-0353f77e2b5d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:18.967075 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01a390-dc36-423a-9b2c-0353f77e2b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:18.967636 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01a390-dc36-423a-9b2c-0353f77e2b5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:18.981350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01a390-dc36-423a-9b2c-0353f77e2b5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:18.987268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-fa01a390-dc36-423a-9b2c-0353f77e2b5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:26:18.987268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:18.987268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:18.987268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:18.987268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:18.987268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:18.987268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:18.987268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:18.987268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:18.987268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:18.987268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:18.987268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:18.987268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:18.987268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:18.987268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:18.987268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:18.988873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:18.988873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:18.988873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:18.988873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:18.988873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:18.988873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:18.988873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:18.988873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:18.988873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:18.988873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:18.988873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:18.988873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:18.988873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:18.988873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:18.988873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:18.988873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:18.990054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:18.990054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:18.990054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:18.990054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:18.990054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:18.990054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:18.990054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:18.990054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:18.990054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:18.990054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:18.990054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:26:18.990054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:18.993799 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 204/401] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:26:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:18.999469 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01a390-dc36-423a-9b2c-0353f77e2b5d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:19.049914 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:19.098768 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800afda-bcf6-4518-b8ab-8b16ab09da2c None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:19.120009 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01a390-dc36-423a-9b2c-0353f77e2b5d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:19.130570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:19.143345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcfff17-2618-4772-8a49-c4edff25b591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:19.144028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcfff17-2618-4772-8a49-c4edff25b591 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:19.144309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcfff17-2618-4772-8a49-c4edff25b591 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:19.144582 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcfff17-2618-4772-8a49-c4edff25b591 None admin] PATH_INFO: `/v3/groups/01077af3c3244f3ab99a742d00bcf1b7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:19.146434 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcfff17-2618-4772-8a49-c4edff25b591 None admin] RBAC: Authorizing `identity:get_group(group_id=01077af3c3244f3ab99a742d00bcf1b7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:19.148516 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcfff17-2618-4772-8a49-c4edff25b591 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:19.153622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 198/402] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Aug 4 04:26:18 2022] GET /identity/v3/groups/01077af3c3244f3ab99a742d00bcf1b7 => generated 226 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:19.155292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f3651-e2d9-4a8d-bdfd-399877ed3f72 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:19.155780 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f3651-e2d9-4a8d-bdfd-399877ed3f72 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:19.157409 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f3651-e2d9-4a8d-bdfd-399877ed3f72 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:19.160233 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 199/403] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:19.161957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f3651-e2d9-4a8d-bdfd-399877ed3f72 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:19.170662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81fe2a5-913a-4ba7-bd46-0d7b5227e652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:19.172690 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fe2a5-913a-4ba7-bd46-0d7b5227e652 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:19.172690 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fe2a5-913a-4ba7-bd46-0d7b5227e652 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:19.172690 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fe2a5-913a-4ba7-bd46-0d7b5227e652 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:19.172892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81fe2a5-913a-4ba7-bd46-0d7b5227e652 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:19.173290 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81fe2a5-913a-4ba7-bd46-0d7b5227e652 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:19.174603 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81fe2a5-913a-4ba7-bd46-0d7b5227e652 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:19.186675 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 205/404] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:19.188309 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235821dc-e49c-411c-8136-269e685c891d None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:19.188605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235821dc-e49c-411c-8136-269e685c891d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:19.188883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235821dc-e49c-411c-8136-269e685c891d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:19.216719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-235821dc-e49c-411c-8136-269e685c891d None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:19.218130 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-235821dc-e49c-411c-8136-269e685c891d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:19.307630 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f3651-e2d9-4a8d-bdfd-399877ed3f72 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:19.308687 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 206/405] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:19.310122 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235821dc-e49c-411c-8136-269e685c891d None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:19.360167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:19.372483 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc34e90-84b8-4c66-b260-5e2288dd93f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:19.373894 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc34e90-84b8-4c66-b260-5e2288dd93f3 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:19.374738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc34e90-84b8-4c66-b260-5e2288dd93f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:19.375363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc34e90-84b8-4c66-b260-5e2288dd93f3 None admin] PATH_INFO: `/v3/projects/88724aa044b04819920009d1b5332589` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:19.378449 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc34e90-84b8-4c66-b260-5e2288dd93f3 None admin] RBAC: Authorizing `identity:get_project(project_id=88724aa044b04819920009d1b5332589)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:19.383523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc34e90-84b8-4c66-b260-5e2288dd93f3 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:19.389211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 200/406] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:26:19 2022] GET /identity/v3/projects/88724aa044b04819920009d1b5332589 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:19.391231 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc34e90-84b8-4c66-b260-5e2288dd93f3 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:19.406100 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:19.443934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235821dc-e49c-411c-8136-269e685c891d None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:19.491634 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06557d-1910-4be7-a7d0-3bf59fd81046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:19.492379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06557d-1910-4be7-a7d0-3bf59fd81046 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:19.492664 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06557d-1910-4be7-a7d0-3bf59fd81046 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:19.492942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06557d-1910-4be7-a7d0-3bf59fd81046 None admin] PATH_INFO: `/v3/groups/d3122077a0cb44cea87c6ddef657da70` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:19.494326 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06557d-1910-4be7-a7d0-3bf59fd81046 None admin] RBAC: Authorizing `identity:get_group(group_id=d3122077a0cb44cea87c6ddef657da70)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:19.496640 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06557d-1910-4be7-a7d0-3bf59fd81046 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:19.503063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 207/407] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Aug 4 04:26:19 2022] GET /identity/v3/groups/d3122077a0cb44cea87c6ddef657da70 => generated 229 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:19.505801 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06557d-1910-4be7-a7d0-3bf59fd81046 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:19.553623 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc34e90-84b8-4c66-b260-5e2288dd93f3 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:19.602639 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef69b81-d0a7-49bd-9b25-1e9743d22e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:19.603882 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef69b81-d0a7-49bd-9b25-1e9743d22e67 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:19.604924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef69b81-d0a7-49bd-9b25-1e9743d22e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:19.605418 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef69b81-d0a7-49bd-9b25-1e9743d22e67 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:19.624680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef69b81-d0a7-49bd-9b25-1e9743d22e67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:19.634176 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-bef69b81-d0a7-49bd-9b25-1e9743d22e67 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:26:19.634176 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:19.634176 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:19.634176 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:19.634176 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:19.634176 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:19.634176 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:19.634176 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:19.634176 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:19.634176 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:19.634176 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:19.634176 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:19.634176 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:19.634176 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:19.634176 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:19.634176 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:19.635105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:19.635105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:19.635105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:19.635105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:19.635105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:19.635105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:19.635105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:19.635105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:19.635105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:19.635105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:19.635105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:19.635105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:19.635105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:19.635105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:19.635105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:19.635105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:19.635885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:19.635885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:19.635885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:19.635885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:19.635885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:19.635885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:19.635885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:19.635885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:19.635885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:19.635885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:19.635885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:26:19.635885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:19.638857 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 201/408] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Aug 4 04:26:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:19.639810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef69b81-d0a7-49bd-9b25-1e9743d22e67 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:19.648453 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06557d-1910-4be7-a7d0-3bf59fd81046 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:19.695482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb90384-c30b-481d-8003-5454cf42f585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:19.696312 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb90384-c30b-481d-8003-5454cf42f585 None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:19.696690 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb90384-c30b-481d-8003-5454cf42f585 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:19.697407 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb90384-c30b-481d-8003-5454cf42f585 None admin] PATH_INFO: `/v3/projects/88724aa044b04819920009d1b5332589/groups/01077af3c3244f3ab99a742d00bcf1b7/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:19.700334 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb90384-c30b-481d-8003-5454cf42f585 None admin] RBAC: Authorizing `identity:create_grant(project_id=88724aa044b04819920009d1b5332589, group_id=01077af3c3244f3ab99a742d00bcf1b7, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:19.702428 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb90384-c30b-481d-8003-5454cf42f585 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:19.723919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:19.724203 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 208/409] 10.222.0.47 () {58 vars in 1442 bytes} [Thu Aug 4 04:26:19 2022] PUT /identity/v3/projects/88724aa044b04819920009d1b5332589/groups/01077af3c3244f3ab99a742d00bcf1b7/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:26:19.725815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb90384-c30b-481d-8003-5454cf42f585 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:19.732084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:19.789322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef69b81-d0a7-49bd-9b25-1e9743d22e67 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:19.826878 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9ea45-9afb-4b08-b610-d8ca9eeae010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:19.827720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9ea45-9afb-4b08-b610-d8ca9eeae010 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:19.828080 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9ea45-9afb-4b08-b610-d8ca9eeae010 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:19.828449 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9ea45-9afb-4b08-b610-d8ca9eeae010 None admin] PATH_INFO: `/v3/projects/312c7b7a94774ca2ba3533f4d13a2124` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:19.829832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9ea45-9afb-4b08-b610-d8ca9eeae010 None admin] RBAC: Authorizing `identity:get_project(project_id=312c7b7a94774ca2ba3533f4d13a2124)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:19.831375 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9ea45-9afb-4b08-b610-d8ca9eeae010 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:19.838161 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 202/410] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:26:19 2022] GET /identity/v3/projects/312c7b7a94774ca2ba3533f4d13a2124 => generated 344 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:19.838161 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56acfd6-836b-451c-9c82-e80258f81510 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:19.838161 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56acfd6-836b-451c-9c82-e80258f81510 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:19.838161 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56acfd6-836b-451c-9c82-e80258f81510 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:19.838161 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 203/411] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:19.840430 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56acfd6-836b-451c-9c82-e80258f81510 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:19.871746 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb90384-c30b-481d-8003-5454cf42f585 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:19.940305 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f3440-1f1b-478f-92eb-773cd5ed42bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:19.941219 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f3440-1f1b-478f-92eb-773cd5ed42bb None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:19.942144 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f3440-1f1b-478f-92eb-773cd5ed42bb None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:19.942699 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f3440-1f1b-478f-92eb-773cd5ed42bb None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:19.943694 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f3440-1f1b-478f-92eb-773cd5ed42bb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:19.944146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f3440-1f1b-478f-92eb-773cd5ed42bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:19.946335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f3440-1f1b-478f-92eb-773cd5ed42bb None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:19.959687 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 209/412] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:19.961648 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3e0de-e1f5-40ba-81f7-bb2dc44184c0 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:19.962035 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3e0de-e1f5-40ba-81f7-bb2dc44184c0 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:19.962520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3e0de-e1f5-40ba-81f7-bb2dc44184c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:19.975855 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56acfd6-836b-451c-9c82-e80258f81510 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:19.990139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-feb3e0de-e1f5-40ba-81f7-bb2dc44184c0 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:19.991843 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-feb3e0de-e1f5-40ba-81f7-bb2dc44184c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:20.010010 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:20.014559 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f76096-abfc-48d4-884b-8070a2014b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:20.015987 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f76096-abfc-48d4-884b-8070a2014b84 None admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:20.016498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f76096-abfc-48d4-884b-8070a2014b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:20.017053 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f76096-abfc-48d4-884b-8070a2014b84 None admin] PATH_INFO: `/v3/projects/312c7b7a94774ca2ba3533f4d13a2124/groups/d3122077a0cb44cea87c6ddef657da70/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:20.020038 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f76096-abfc-48d4-884b-8070a2014b84 None admin] RBAC: Authorizing `identity:create_grant(project_id=312c7b7a94774ca2ba3533f4d13a2124, group_id=d3122077a0cb44cea87c6ddef657da70, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:20.022002 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f76096-abfc-48d4-884b-8070a2014b84 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:20.040387 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 204/413] 10.222.0.47 () {58 vars in 1442 bytes} [Thu Aug 4 04:26:19 2022] PUT /identity/v3/projects/312c7b7a94774ca2ba3533f4d13a2124/groups/d3122077a0cb44cea87c6ddef657da70/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:26:20.043544 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f76096-abfc-48d4-884b-8070a2014b84 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:20.115674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:20.132666 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 210/414] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:20.159555 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3e0de-e1f5-40ba-81f7-bb2dc44184c0 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:20.273278 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f76096-abfc-48d4-884b-8070a2014b84 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:20.312093 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b916533-d36f-4edf-9a8e-d018a4a13ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:20.312985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b916533-d36f-4edf-9a8e-d018a4a13ccd None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:20.313260 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b916533-d36f-4edf-9a8e-d018a4a13ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:20.313539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b916533-d36f-4edf-9a8e-d018a4a13ccd None admin] PATH_INFO: `/v3/groups/01077af3c3244f3ab99a742d00bcf1b7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:20.314732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b916533-d36f-4edf-9a8e-d018a4a13ccd None admin] RBAC: Authorizing `identity:get_group(group_id=01077af3c3244f3ab99a742d00bcf1b7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:20.316239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b916533-d36f-4edf-9a8e-d018a4a13ccd None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:20.318196 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 205/415] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Aug 4 04:26:20 2022] GET /identity/v3/groups/01077af3c3244f3ab99a742d00bcf1b7 => generated 226 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:20.327164 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b916533-d36f-4edf-9a8e-d018a4a13ccd None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:20.328962 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3e0de-e1f5-40ba-81f7-bb2dc44184c0 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:20.381564 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646909ca-80ef-42d6-9b26-b63b3308de0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:20.382220 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646909ca-80ef-42d6-9b26-b63b3308de0e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:20.382501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646909ca-80ef-42d6-9b26-b63b3308de0e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:20.382825 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646909ca-80ef-42d6-9b26-b63b3308de0e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:20.406341 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646909ca-80ef-42d6-9b26-b63b3308de0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:20.412372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-646909ca-80ef-42d6-9b26-b63b3308de0e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:26:20.412372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:20.412372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:20.412372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:20.412372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:20.412372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:20.412372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:20.412372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:20.412372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:20.412372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:20.412372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:20.412372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:20.412372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:20.412372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:20.412372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:20.412372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:20.413221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:20.413221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:20.413221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:20.413221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:20.413221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:20.413221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:20.413221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:20.413221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:20.413221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:20.413221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:20.413221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:20.413221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:20.413221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:20.413221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:20.413221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:20.413221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:20.414022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:20.414022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:20.414022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:20.414022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:20.414022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:20.414022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:20.414022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:20.414022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:20.414022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:20.414022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:20.414022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:26:20.414022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:20.414867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 211/416] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Aug 4 04:26:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:26:20.423368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646909ca-80ef-42d6-9b26-b63b3308de0e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:20.437924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:20.502177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b916533-d36f-4edf-9a8e-d018a4a13ccd None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:20.512229 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:20.542071 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23dee3c-db83-4d4b-b61a-6bbb56c31b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:20.542829 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23dee3c-db83-4d4b-b61a-6bbb56c31b43 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:20.543118 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23dee3c-db83-4d4b-b61a-6bbb56c31b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:20.543394 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23dee3c-db83-4d4b-b61a-6bbb56c31b43 None admin] PATH_INFO: `/v3/projects/88724aa044b04819920009d1b5332589` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:20.544628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23dee3c-db83-4d4b-b61a-6bbb56c31b43 None admin] RBAC: Authorizing `identity:get_project(project_id=88724aa044b04819920009d1b5332589)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:20.546644 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23dee3c-db83-4d4b-b61a-6bbb56c31b43 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:20.548657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 206/417] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:26:20 2022] GET /identity/v3/projects/88724aa044b04819920009d1b5332589 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:20.550016 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131659c-591c-4152-af89-9084037089e1 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:20.550332 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131659c-591c-4152-af89-9084037089e1 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:20.551245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131659c-591c-4152-af89-9084037089e1 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:20.556889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 207/418] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:20.558523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57dcd9-f8e1-4dac-a3fa-d622b25b5bda None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:20.558811 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57dcd9-f8e1-4dac-a3fa-d622b25b5bda None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:20.559103 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57dcd9-f8e1-4dac-a3fa-d622b25b5bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:20.569886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646909ca-80ef-42d6-9b26-b63b3308de0e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:20.578738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-ec57dcd9-f8e1-4dac-a3fa-d622b25b5bda None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:20.579955 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-ec57dcd9-f8e1-4dac-a3fa-d622b25b5bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:20.610231 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de1442-d377-4314-a8b8-69d67c612eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:20.610954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de1442-d377-4314-a8b8-69d67c612eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:20.611240 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de1442-d377-4314-a8b8-69d67c612eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:20.611513 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de1442-d377-4314-a8b8-69d67c612eb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:20.612337 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de1442-d377-4314-a8b8-69d67c612eb2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:20.612648 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de1442-d377-4314-a8b8-69d67c612eb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:20.615170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de1442-d377-4314-a8b8-69d67c612eb2 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:20.624476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 212/419] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:20.625284 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de1442-d377-4314-a8b8-69d67c612eb2 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:20.669916 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 208/420] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:20.670777 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57dcd9-f8e1-4dac-a3fa-d622b25b5bda None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:20.739659 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de1442-d377-4314-a8b8-69d67c612eb2 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:20.750642 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:20.760366 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:20.779532 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752d36d-c6bc-4c0a-b8b7-000903e4a0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:20.780190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752d36d-c6bc-4c0a-b8b7-000903e4a0ad None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:20.780479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752d36d-c6bc-4c0a-b8b7-000903e4a0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:20.780753 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752d36d-c6bc-4c0a-b8b7-000903e4a0ad None admin] PATH_INFO: `/v3/projects/88724aa044b04819920009d1b5332589/groups/01077af3c3244f3ab99a742d00bcf1b7/roles/7a855ec41b944458821d76509b481268` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:20.783496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752d36d-c6bc-4c0a-b8b7-000903e4a0ad None admin] RBAC: Authorizing `identity:create_grant(project_id=88724aa044b04819920009d1b5332589, group_id=01077af3c3244f3ab99a742d00bcf1b7, role_id=7a855ec41b944458821d76509b481268)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:20.786253 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752d36d-c6bc-4c0a-b8b7-000903e4a0ad None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:20.804258 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 213/421] 10.222.0.47 () {58 vars in 1442 bytes} [Thu Aug 4 04:26:20 2022] PUT /identity/v3/projects/88724aa044b04819920009d1b5332589/groups/01077af3c3244f3ab99a742d00bcf1b7/roles/7a855ec41b944458821d76509b481268 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:26:20.805352 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752d36d-c6bc-4c0a-b8b7-000903e4a0ad None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:20.810886 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57dcd9-f8e1-4dac-a3fa-d622b25b5bda None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:20.868933 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4972e3f-1e66-442d-82f6-663c77773d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:20.870025 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4972e3f-1e66-442d-82f6-663c77773d2e None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:20.870513 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4972e3f-1e66-442d-82f6-663c77773d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:20.870994 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4972e3f-1e66-442d-82f6-663c77773d2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:20.930180 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752d36d-c6bc-4c0a-b8b7-000903e4a0ad None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:20.960824 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4972e3f-1e66-442d-82f6-663c77773d2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:20.962770 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4972e3f-1e66-442d-82f6-663c77773d2e None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:20.968741 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded4ae08-e3de-40ae-97fb-1131bb6b2b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:20.969443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4ae08-e3de-40ae-97fb-1131bb6b2b73 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:20.969759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4ae08-e3de-40ae-97fb-1131bb6b2b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:20.970078 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4ae08-e3de-40ae-97fb-1131bb6b2b73 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:20.982576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded4ae08-e3de-40ae-97fb-1131bb6b2b73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:20.989209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-ded4ae08-e3de-40ae-97fb-1131bb6b2b73 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:26:20.989209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:20.989209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:20.989209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:20.989209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:20.989209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:20.989209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:20.989209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:20.989209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:20.989209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:20.989209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:20.989209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:20.989209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:20.989209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:20.989209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:20.989209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:20.990191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:20.990191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:20.990191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:20.990191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:20.990191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:20.990191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:20.990191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:20.990191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:20.990191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:20.990191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:20.990191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:20.990191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:20.990191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:20.990191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:20.990191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:20.990191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:20.990973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:20.990973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:20.990973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:20.990973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:20.990973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:20.990973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:20.990973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:20.990973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:20.990973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:20.990973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:20.990973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:26:20.990973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:20.990973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 214/422] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:20.999396 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded4ae08-e3de-40ae-97fb-1131bb6b2b73 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:21.042631 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:21.046678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:21.123069 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 209/423] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:26:21.130315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4972e3f-1e66-442d-82f6-663c77773d2e None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:21.146626 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded4ae08-e3de-40ae-97fb-1131bb6b2b73 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:21.196981 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20baedd0-5252-478d-aa62-850bd7b6fbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:21.197533 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20baedd0-5252-478d-aa62-850bd7b6fbe0 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:21.197845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20baedd0-5252-478d-aa62-850bd7b6fbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:21.198118 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20baedd0-5252-478d-aa62-850bd7b6fbe0 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:21.198853 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20baedd0-5252-478d-aa62-850bd7b6fbe0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:21.199160 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20baedd0-5252-478d-aa62-850bd7b6fbe0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:21.200556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20baedd0-5252-478d-aa62-850bd7b6fbe0 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:21.216986 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 215/424] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:26:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:21.222971 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20baedd0-5252-478d-aa62-850bd7b6fbe0 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:21.255557 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4972e3f-1e66-442d-82f6-663c77773d2e None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:21.292698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db818193-2fe2-437e-b4bf-3e55f75e9a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:21.293442 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db818193-2fe2-437e-b4bf-3e55f75e9a59 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:21.293705 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db818193-2fe2-437e-b4bf-3e55f75e9a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:21.293987 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db818193-2fe2-437e-b4bf-3e55f75e9a59 None admin] PATH_INFO: `/v3/projects/ba9c9c811ec54130a5b53525b32708d8` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:21.295449 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db818193-2fe2-437e-b4bf-3e55f75e9a59 None admin] RBAC: Authorizing `identity:get_project(project_id=ba9c9c811ec54130a5b53525b32708d8)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:21.297157 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db818193-2fe2-437e-b4bf-3e55f75e9a59 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:21.299360 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 210/425] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:26:21 2022] GET /identity/v3/projects/ba9c9c811ec54130a5b53525b32708d8 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:21.314534 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db818193-2fe2-437e-b4bf-3e55f75e9a59 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:21.353538 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:21.356828 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20baedd0-5252-478d-aa62-850bd7b6fbe0 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:21.362429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:21.395706 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59efb0e3-1f20-4b18-8823-0f8d6c7de26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:21.396501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efb0e3-1f20-4b18-8823-0f8d6c7de26c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:21.396860 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efb0e3-1f20-4b18-8823-0f8d6c7de26c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:21.397205 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efb0e3-1f20-4b18-8823-0f8d6c7de26c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:21.471048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59efb0e3-1f20-4b18-8823-0f8d6c7de26c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:21.473229 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59efb0e3-1f20-4b18-8823-0f8d6c7de26c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:21.550113 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db818193-2fe2-437e-b4bf-3e55f75e9a59 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:21.593730 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 216/426] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 04:26:21.594427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:21.597639 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f1218-1e74-41a2-8ded-a15dad64123e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:21.598291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f1218-1e74-41a2-8ded-a15dad64123e None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:21.598567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f1218-1e74-41a2-8ded-a15dad64123e None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:21.598839 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f1218-1e74-41a2-8ded-a15dad64123e None admin] PATH_INFO: `/v3/groups/01077af3c3244f3ab99a742d00bcf1b7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:21.600025 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f1218-1e74-41a2-8ded-a15dad64123e None admin] RBAC: Authorizing `identity:get_group(group_id=01077af3c3244f3ab99a742d00bcf1b7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:21.601584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f1218-1e74-41a2-8ded-a15dad64123e None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:21.601783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59efb0e3-1f20-4b18-8823-0f8d6c7de26c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:21.603518 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 211/427] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Aug 4 04:26:21 2022] GET /identity/v3/groups/01077af3c3244f3ab99a742d00bcf1b7 => generated 226 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:21.610076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f1218-1e74-41a2-8ded-a15dad64123e None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:21.724126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:21.735219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f1218-1e74-41a2-8ded-a15dad64123e None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:21.773681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b4e54-cce1-4548-ba7f-6dc1b8af2063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:21.774303 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b4e54-cce1-4548-ba7f-6dc1b8af2063 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:21.774580 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b4e54-cce1-4548-ba7f-6dc1b8af2063 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:21.774971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b4e54-cce1-4548-ba7f-6dc1b8af2063 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:21.776240 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b4e54-cce1-4548-ba7f-6dc1b8af2063 None admin] RBAC: Adding query filter params (group.id=01077af3c3244f3ab99a742d00bcf1b7, role.id=7a855ec41b944458821d76509b481268, scope.project.id=ba9c9c811ec54130a5b53525b32708d8) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:21.776550 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b4e54-cce1-4548-ba7f-6dc1b8af2063 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:21.778062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b4e54-cce1-4548-ba7f-6dc1b8af2063 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:21.785733 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 212/428] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Aug 4 04:26:21 2022] GET /identity/v3/role_assignments?group.id=01077af3c3244f3ab99a742d00bcf1b7&scope.project.id=ba9c9c811ec54130a5b53525b32708d8&role.id=7a855ec41b944458821d76509b481268 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:21.814230 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59efb0e3-1f20-4b18-8823-0f8d6c7de26c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:21.862361 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd158c47-4349-4267-8486-b6ebcb1d93ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:21.863105 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd158c47-4349-4267-8486-b6ebcb1d93ff None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:21.863385 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd158c47-4349-4267-8486-b6ebcb1d93ff None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:21.863855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd158c47-4349-4267-8486-b6ebcb1d93ff None admin] PATH_INFO: `/v3/projects/ba9c9c811ec54130a5b53525b32708d8` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:21.865153 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd158c47-4349-4267-8486-b6ebcb1d93ff None admin] RBAC: Authorizing `identity:get_project(project_id=ba9c9c811ec54130a5b53525b32708d8)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:21.866628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd158c47-4349-4267-8486-b6ebcb1d93ff None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:21.868744 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 217/429] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:26:21 2022] GET /identity/v3/projects/ba9c9c811ec54130a5b53525b32708d8 => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:21.874191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b4e54-cce1-4548-ba7f-6dc1b8af2063 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:22.023034 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b4e54-cce1-4548-ba7f-6dc1b8af2063 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:22.071789 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca4fb3-fc17-44b2-b778-500747188eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:22.072497 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca4fb3-fc17-44b2-b778-500747188eda None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:22.072774 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca4fb3-fc17-44b2-b778-500747188eda None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:22.073043 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca4fb3-fc17-44b2-b778-500747188eda None admin] PATH_INFO: `/v3/groups/01077af3c3244f3ab99a742d00bcf1b7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:22.075262 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca4fb3-fc17-44b2-b778-500747188eda None admin] RBAC: Authorizing `identity:get_group(group_id=01077af3c3244f3ab99a742d00bcf1b7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:22.076691 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca4fb3-fc17-44b2-b778-500747188eda None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:22.078310 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 213/430] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Aug 4 04:26:21 2022] GET /identity/v3/groups/01077af3c3244f3ab99a742d00bcf1b7 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:22.084588 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd158c47-4349-4267-8486-b6ebcb1d93ff None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:22.173333 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:22.212041 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd158c47-4349-4267-8486-b6ebcb1d93ff None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:22.217170 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47733742-7a4e-49ef-a980-14a71d4ee1e9 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:22.217462 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47733742-7a4e-49ef-a980-14a71d4ee1e9 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:22.217734 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47733742-7a4e-49ef-a980-14a71d4ee1e9 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:22.219084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 214/431] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:26:22.226500 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150cf07-75c2-4f02-a01c-51277b1b6b95 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:22.226500 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150cf07-75c2-4f02-a01c-51277b1b6b95 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:22.226878 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150cf07-75c2-4f02-a01c-51277b1b6b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:22.255837 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-3150cf07-75c2-4f02-a01c-51277b1b6b95 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:22.257427 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-3150cf07-75c2-4f02-a01c-51277b1b6b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:22.263758 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf6718-2d58-48eb-8260-dd4a92e7adb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:22.264388 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf6718-2d58-48eb-8260-dd4a92e7adb7 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:22.264671 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf6718-2d58-48eb-8260-dd4a92e7adb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:22.264949 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf6718-2d58-48eb-8260-dd4a92e7adb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:22.265810 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf6718-2d58-48eb-8260-dd4a92e7adb7 None admin] RBAC: Adding query filter params (group.id=01077af3c3244f3ab99a742d00bcf1b7, role.id=e733372bae6047be956ad703f83a480e, scope.project.id=ba9c9c811ec54130a5b53525b32708d8) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:22.266124 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf6718-2d58-48eb-8260-dd4a92e7adb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:22.267771 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf6718-2d58-48eb-8260-dd4a92e7adb7 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:22.278514 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 218/432] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Aug 4 04:26:22 2022] GET /identity/v3/role_assignments?group.id=01077af3c3244f3ab99a742d00bcf1b7&scope.project.id=ba9c9c811ec54130a5b53525b32708d8&role.id=e733372bae6047be956ad703f83a480e => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:22.297966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:22.365048 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 215/433] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:22.389809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf6718-2d58-48eb-8260-dd4a92e7adb7 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:22.522619 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf6718-2d58-48eb-8260-dd4a92e7adb7 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:22.554855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:22.560292 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0e222-07f7-428d-8170-f18931be736f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:22.561812 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0e222-07f7-428d-8170-f18931be736f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:22.561812 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0e222-07f7-428d-8170-f18931be736f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:22.561812 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0e222-07f7-428d-8170-f18931be736f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:22.575077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0e222-07f7-428d-8170-f18931be736f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:22.580815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-c5f0e222-07f7-428d-8170-f18931be736f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:26:22.580815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:22.580815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:22.580815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:22.580815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:22.580815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:22.580815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:22.580815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:22.580815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:22.580815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:22.580815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:22.580815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:22.580815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:22.580815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:22.580815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:22.580815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:22.582522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:22.582522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:22.582522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:22.582522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:22.582522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:22.582522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:22.582522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:22.582522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:22.582522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:22.582522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:22.582522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:22.582522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:22.582522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:22.582522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:22.582522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:22.582522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:22.583325 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:22.583325 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:22.583325 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:22.583325 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:22.583325 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:22.583325 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:22.583325 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:22.583325 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:22.583325 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:22.583325 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:22.583325 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 04:26:22.583325 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:22.583325 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 219/434] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:22.589822 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150cf07-75c2-4f02-a01c-51277b1b6b95 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:22.616723 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6fdc7-4c5e-4db9-ae0f-ecec14fd7c14 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:22.617064 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6fdc7-4c5e-4db9-ae0f-ecec14fd7c14 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:22.617344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6fdc7-4c5e-4db9-ae0f-ecec14fd7c14 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:22.619006 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 220/435] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:26:22.625763 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a1bf4-ee6a-4fb8-aeac-203204eb6ece None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:22.626203 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a1bf4-ee6a-4fb8-aeac-203204eb6ece None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:22.626539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a1bf4-ee6a-4fb8-aeac-203204eb6ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:22.647883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-968a1bf4-ee6a-4fb8-aeac-203204eb6ece None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:22.649173 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-968a1bf4-ee6a-4fb8-aeac-203204eb6ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:22.709467 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150cf07-75c2-4f02-a01c-51277b1b6b95 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:22.727879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 221/436] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:22.751285 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a1bf4-ee6a-4fb8-aeac-203204eb6ece None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:22.758812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094fe0f-83c8-4ea9-809d-97e8512bc5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:22.759464 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094fe0f-83c8-4ea9-809d-97e8512bc5f6 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:22.759733 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094fe0f-83c8-4ea9-809d-97e8512bc5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:22.760007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094fe0f-83c8-4ea9-809d-97e8512bc5f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:22.760776 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094fe0f-83c8-4ea9-809d-97e8512bc5f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:22.761082 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094fe0f-83c8-4ea9-809d-97e8512bc5f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:22.762516 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094fe0f-83c8-4ea9-809d-97e8512bc5f6 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:22.776551 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 216/437] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:26:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:22.782256 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094fe0f-83c8-4ea9-809d-97e8512bc5f6 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:22.802722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:22.904852 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a1bf4-ee6a-4fb8-aeac-203204eb6ece None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:22.924624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094fe0f-83c8-4ea9-809d-97e8512bc5f6 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:22.942001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:22.965028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099a5ce-0738-497f-9955-87990f8d1c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:22.965028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099a5ce-0738-497f-9955-87990f8d1c9c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:22.965028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099a5ce-0738-497f-9955-87990f8d1c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:22.965372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099a5ce-0738-497f-9955-87990f8d1c9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:22.977126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c41bf-446e-4c41-a20e-16244cf04047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:22.977800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c41bf-446e-4c41-a20e-16244cf04047 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:22.978080 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c41bf-446e-4c41-a20e-16244cf04047 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:22.978348 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c41bf-446e-4c41-a20e-16244cf04047 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:22.979610 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0099a5ce-0738-497f-9955-87990f8d1c9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:22.985066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-0099a5ce-0738-497f-9955-87990f8d1c9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:26:22.985066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:22.985066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:22.985066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:22.985066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:22.985066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:22.985066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:22.985066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:22.985066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:22.985066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:22.985066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:22.985066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:22.985066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:22.985066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:22.985066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:22.985066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:22.986057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:22.986057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:22.986057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:22.986057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:22.986057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:22.986057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:22.986057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:22.986057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:22.986057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:22.986057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:22.986057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:22.986057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:22.986057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:22.986057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:22.986057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:22.986057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:22.986833 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:22.986833 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:22.986833 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:22.986833 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:22.986833 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:22.986833 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:22.986833 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:22.986833 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:22.986833 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:22.986833 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:22.986833 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:26:22.986833 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:22.992659 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 222/438] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:26:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:26:22.994083 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099a5ce-0738-497f-9955-87990f8d1c9c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:23.067161 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c41bf-446e-4c41-a20e-16244cf04047 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:23.068638 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c41bf-446e-4c41-a20e-16244cf04047 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:23.088934 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:23.114704 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099a5ce-0738-497f-9955-87990f8d1c9c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:23.152662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e264e92-fc29-44b1-9dbc-c9f4ecc2bc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:23.153277 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e264e92-fc29-44b1-9dbc-c9f4ecc2bc07 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:23.153570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e264e92-fc29-44b1-9dbc-c9f4ecc2bc07 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:23.153972 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e264e92-fc29-44b1-9dbc-c9f4ecc2bc07 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:23.154700 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e264e92-fc29-44b1-9dbc-c9f4ecc2bc07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:23.155060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e264e92-fc29-44b1-9dbc-c9f4ecc2bc07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:23.156504 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e264e92-fc29-44b1-9dbc-c9f4ecc2bc07 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:23.170517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 223/439] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:23.171801 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e264e92-fc29-44b1-9dbc-c9f4ecc2bc07 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:23.184483 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 217/440] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 04:26:23.189906 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c41bf-446e-4c41-a20e-16244cf04047 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:23.219809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:23.277776 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e264e92-fc29-44b1-9dbc-c9f4ecc2bc07 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:23.295181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c41bf-446e-4c41-a20e-16244cf04047 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:23.310169 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:23.318001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5557f6-da3e-4ea1-a262-3c291fa4784d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:23.318632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5557f6-da3e-4ea1-a262-3c291fa4784d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:23.318915 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5557f6-da3e-4ea1-a262-3c291fa4784d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:23.319190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5557f6-da3e-4ea1-a262-3c291fa4784d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:23.332699 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118a47d-28c1-46e5-b02c-8eea69fbeabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:23.333773 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118a47d-28c1-46e5-b02c-8eea69fbeabb None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:23.334012 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118a47d-28c1-46e5-b02c-8eea69fbeabb None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:23.334290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118a47d-28c1-46e5-b02c-8eea69fbeabb None admin] PATH_INFO: `/v3/projects/88724aa044b04819920009d1b5332589` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:23.335472 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f118a47d-28c1-46e5-b02c-8eea69fbeabb None admin] RBAC: Authorizing `identity:get_project(project_id=88724aa044b04819920009d1b5332589)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:23.336886 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f118a47d-28c1-46e5-b02c-8eea69fbeabb None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:23.338627 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 218/441] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:26:23 2022] GET /identity/v3/projects/88724aa044b04819920009d1b5332589 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:23.344168 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118a47d-28c1-46e5-b02c-8eea69fbeabb None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:23.396623 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5557f6-da3e-4ea1-a262-3c291fa4784d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:23.398119 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5557f6-da3e-4ea1-a262-3c291fa4784d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:23.442083 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:23.447296 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118a47d-28c1-46e5-b02c-8eea69fbeabb None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:23.481592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a01ab-a833-46bb-9da8-c710db8cd974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:23.482227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a01ab-a833-46bb-9da8-c710db8cd974 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:23.482500 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a01ab-a833-46bb-9da8-c710db8cd974 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:23.482771 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a01ab-a833-46bb-9da8-c710db8cd974 None admin] PATH_INFO: `/v3/groups/01077af3c3244f3ab99a742d00bcf1b7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:23.483912 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a01ab-a833-46bb-9da8-c710db8cd974 None admin] RBAC: Authorizing `identity:get_group(group_id=01077af3c3244f3ab99a742d00bcf1b7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:23.485333 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a01ab-a833-46bb-9da8-c710db8cd974 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:23.486922 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 219/442] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Aug 4 04:26:23 2022] GET /identity/v3/groups/01077af3c3244f3ab99a742d00bcf1b7 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:23.488100 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ab02f-1183-4add-ab1c-2d5e3ed04034 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:23.488559 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ab02f-1183-4add-ab1c-2d5e3ed04034 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:23.488694 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ab02f-1183-4add-ab1c-2d5e3ed04034 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:23.492893 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 220/443] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:23.492893 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ab02f-1183-4add-ab1c-2d5e3ed04034 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:23.501581 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 224/444] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:26:23.502686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56a3a1-f474-4d35-9e55-b9dd201d4161 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:23.502971 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56a3a1-f474-4d35-9e55-b9dd201d4161 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:23.503796 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56a3a1-f474-4d35-9e55-b9dd201d4161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:23.525195 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-2b56a3a1-f474-4d35-9e55-b9dd201d4161 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:23.526398 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2b56a3a1-f474-4d35-9e55-b9dd201d4161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:23.548649 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:23.594666 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 225/445] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:23.595516 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56a3a1-f474-4d35-9e55-b9dd201d4161 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:23.608771 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ab02f-1183-4add-ab1c-2d5e3ed04034 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:23.642680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d23a27-e3a1-4465-9a69-9ef734bf3ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:23.643284 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d23a27-e3a1-4465-9a69-9ef734bf3ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:23.643555 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d23a27-e3a1-4465-9a69-9ef734bf3ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:23.644491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d23a27-e3a1-4465-9a69-9ef734bf3ae6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:23.645258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d23a27-e3a1-4465-9a69-9ef734bf3ae6 None admin] RBAC: Adding query filter params (group.id=01077af3c3244f3ab99a742d00bcf1b7, role.id=e733372bae6047be956ad703f83a480e, scope.project.id=88724aa044b04819920009d1b5332589) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:23.645579 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d23a27-e3a1-4465-9a69-9ef734bf3ae6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:23.646984 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d23a27-e3a1-4465-9a69-9ef734bf3ae6 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:23.652680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 221/446] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Aug 4 04:26:23 2022] GET /identity/v3/role_assignments?group.id=01077af3c3244f3ab99a742d00bcf1b7&scope.project.id=88724aa044b04819920009d1b5332589&role.id=e733372bae6047be956ad703f83a480e => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:23.658560 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d23a27-e3a1-4465-9a69-9ef734bf3ae6 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:23.673459 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:23.718220 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56a3a1-f474-4d35-9e55-b9dd201d4161 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:23.759065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d051a85-ea9f-4800-bebe-3aaa2a18b421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:23.759734 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d051a85-ea9f-4800-bebe-3aaa2a18b421 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:23.760019 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d051a85-ea9f-4800-bebe-3aaa2a18b421 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:23.760296 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d051a85-ea9f-4800-bebe-3aaa2a18b421 None admin] PATH_INFO: `/v3/projects/312c7b7a94774ca2ba3533f4d13a2124` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:23.761506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d051a85-ea9f-4800-bebe-3aaa2a18b421 None admin] RBAC: Authorizing `identity:get_project(project_id=312c7b7a94774ca2ba3533f4d13a2124)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:23.763025 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d051a85-ea9f-4800-bebe-3aaa2a18b421 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:23.764744 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 226/447] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:26:23 2022] GET /identity/v3/projects/312c7b7a94774ca2ba3533f4d13a2124 => generated 344 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:23.771657 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d051a85-ea9f-4800-bebe-3aaa2a18b421 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:23.791306 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d23a27-e3a1-4465-9a69-9ef734bf3ae6 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:23.807118 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:23.834512 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479544d-63bf-4c75-9c27-b80cfd1b946e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:23.835337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479544d-63bf-4c75-9c27-b80cfd1b946e None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:23.835618 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479544d-63bf-4c75-9c27-b80cfd1b946e None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:23.835889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479544d-63bf-4c75-9c27-b80cfd1b946e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:23.848043 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479544d-63bf-4c75-9c27-b80cfd1b946e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:23.854388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-6479544d-63bf-4c75-9c27-b80cfd1b946e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:26:23.854388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:23.854388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:23.854388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:23.854388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:23.854388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:23.854388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:23.854388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:23.854388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:23.854388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:23.854388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:23.854388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:23.854388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:23.854388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:23.854388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:23.854388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:23.855277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:23.855277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:23.855277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:23.855277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:23.855277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:23.855277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:23.855277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:23.855277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:23.855277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:23.855277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:23.855277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:23.855277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:23.855277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:23.855277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:23.855277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:23.855277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:23.856037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:23.856037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:23.856037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:23.856037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:23.856037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:23.856037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:23.856037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:23.856037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:23.856037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:23.856037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:23.856037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:26:23.856037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:23.856037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 222/448] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Aug 4 04:26:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:23.862379 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479544d-63bf-4c75-9c27-b80cfd1b946e None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:23.884185 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d051a85-ea9f-4800-bebe-3aaa2a18b421 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:23.922893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2b061-616a-4312-8e61-53352d145595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:23.923537 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2b061-616a-4312-8e61-53352d145595 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:23.923813 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2b061-616a-4312-8e61-53352d145595 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:23.924086 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2b061-616a-4312-8e61-53352d145595 None admin] PATH_INFO: `/v3/groups/d3122077a0cb44cea87c6ddef657da70` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:23.925357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2b061-616a-4312-8e61-53352d145595 None admin] RBAC: Authorizing `identity:get_group(group_id=d3122077a0cb44cea87c6ddef657da70)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:23.926930 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2b061-616a-4312-8e61-53352d145595 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:23.928528 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 227/449] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Aug 4 04:26:23 2022] GET /identity/v3/groups/d3122077a0cb44cea87c6ddef657da70 => generated 229 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:23.934356 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2b061-616a-4312-8e61-53352d145595 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:23.955287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:23.999958 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479544d-63bf-4c75-9c27-b80cfd1b946e None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:24.036014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c75bbbf-a929-4670-bb74-de65de9b4df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:24.037447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75bbbf-a929-4670-bb74-de65de9b4df4 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:24.037778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75bbbf-a929-4670-bb74-de65de9b4df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:24.038056 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75bbbf-a929-4670-bb74-de65de9b4df4 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:24.038786 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c75bbbf-a929-4670-bb74-de65de9b4df4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:24.039090 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c75bbbf-a929-4670-bb74-de65de9b4df4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:24.040461 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c75bbbf-a929-4670-bb74-de65de9b4df4 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:24.049055 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 223/450] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:24.056073 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c75bbbf-a929-4670-bb74-de65de9b4df4 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:24.067933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2b061-616a-4312-8e61-53352d145595 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:24.092796 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:24.103029 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1cd3e-65df-4ff1-a83a-f1bb94115037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:24.103678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1cd3e-65df-4ff1-a83a-f1bb94115037 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:24.103957 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1cd3e-65df-4ff1-a83a-f1bb94115037 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:24.104255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1cd3e-65df-4ff1-a83a-f1bb94115037 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:24.105040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1cd3e-65df-4ff1-a83a-f1bb94115037 None admin] RBAC: Adding query filter params (group.id=d3122077a0cb44cea87c6ddef657da70, role.id=4a6517e5890a49718303007dac0756c7, scope.project.id=312c7b7a94774ca2ba3533f4d13a2124) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:24.105344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1cd3e-65df-4ff1-a83a-f1bb94115037 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:24.106798 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1cd3e-65df-4ff1-a83a-f1bb94115037 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:24.112564 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 228/451] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Aug 4 04:26:23 2022] GET /identity/v3/role_assignments?group.id=d3122077a0cb44cea87c6ddef657da70&scope.project.id=312c7b7a94774ca2ba3533f4d13a2124&role.id=4a6517e5890a49718303007dac0756c7 => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:24.161225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c75bbbf-a929-4670-bb74-de65de9b4df4 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:24.197310 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8b65d-e55c-47f2-bee2-442cba573188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:24.197985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8b65d-e55c-47f2-bee2-442cba573188 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:24.198262 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8b65d-e55c-47f2-bee2-442cba573188 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:24.198531 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8b65d-e55c-47f2-bee2-442cba573188 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:24.270230 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba8b65d-e55c-47f2-bee2-442cba573188 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:24.271659 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba8b65d-e55c-47f2-bee2-442cba573188 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:24.306241 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:24.371943 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 224/452] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 04:26:24.377624 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1cd3e-65df-4ff1-a83a-f1bb94115037 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:24.450691 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:24.476443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1cd3e-65df-4ff1-a83a-f1bb94115037 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:24.509746 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef4c9b-22fd-4d50-9f09-69fcfb20ef4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:24.510382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef4c9b-22fd-4d50-9f09-69fcfb20ef4e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:24.510657 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef4c9b-22fd-4d50-9f09-69fcfb20ef4e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:24.510926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef4c9b-22fd-4d50-9f09-69fcfb20ef4e None admin] PATH_INFO: `/v3/projects/88724aa044b04819920009d1b5332589` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:24.512075 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef4c9b-22fd-4d50-9f09-69fcfb20ef4e None admin] RBAC: Authorizing `identity:get_project(project_id=88724aa044b04819920009d1b5332589)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:24.513526 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef4c9b-22fd-4d50-9f09-69fcfb20ef4e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:24.515181 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 229/453] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:26:24 2022] GET /identity/v3/projects/88724aa044b04819920009d1b5332589 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:24.520917 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8b65d-e55c-47f2-bee2-442cba573188 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:24.618725 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8b65d-e55c-47f2-bee2-442cba573188 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:24.654442 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631dba4-1fd1-4d54-b189-79e327209f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:24.655077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631dba4-1fd1-4d54-b189-79e327209f1c None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:24.655353 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631dba4-1fd1-4d54-b189-79e327209f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:24.655625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631dba4-1fd1-4d54-b189-79e327209f1c None admin] PATH_INFO: `/v3/groups/01077af3c3244f3ab99a742d00bcf1b7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:24.656762 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b631dba4-1fd1-4d54-b189-79e327209f1c None admin] RBAC: Authorizing `identity:get_group(group_id=01077af3c3244f3ab99a742d00bcf1b7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:24.658221 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b631dba4-1fd1-4d54-b189-79e327209f1c None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:24.660069 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 225/454] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Aug 4 04:26:24 2022] GET /identity/v3/groups/01077af3c3244f3ab99a742d00bcf1b7 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:24.665922 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef4c9b-22fd-4d50-9f09-69fcfb20ef4e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:24.761479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef4c9b-22fd-4d50-9f09-69fcfb20ef4e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:24.792880 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f89fb9-c247-4a34-9930-15ed6cbd115d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:24.793473 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f89fb9-c247-4a34-9930-15ed6cbd115d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:24.793778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f89fb9-c247-4a34-9930-15ed6cbd115d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:24.794046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f89fb9-c247-4a34-9930-15ed6cbd115d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:24.794823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f89fb9-c247-4a34-9930-15ed6cbd115d None admin] RBAC: Adding query filter params (group.id=01077af3c3244f3ab99a742d00bcf1b7, role.id=7a855ec41b944458821d76509b481268, scope.project.id=88724aa044b04819920009d1b5332589) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:24.795128 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f89fb9-c247-4a34-9930-15ed6cbd115d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:24.796808 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f89fb9-c247-4a34-9930-15ed6cbd115d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:24.802873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 230/455] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Aug 4 04:26:24 2022] GET /identity/v3/role_assignments?group.id=01077af3c3244f3ab99a742d00bcf1b7&scope.project.id=88724aa044b04819920009d1b5332589&role.id=7a855ec41b944458821d76509b481268 => generated 611 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:27.477008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227bde0-9c54-4382-9608-7c46d4d21946 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:27.477538 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227bde0-9c54-4382-9608-7c46d4d21946 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:27.477538 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227bde0-9c54-4382-9608-7c46d4d21946 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:27.479159 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 226/456] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:27.484514 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda9566-4af1-41a8-923b-0c882f404a07 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:27.485334 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda9566-4af1-41a8-923b-0c882f404a07 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:27.485847 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda9566-4af1-41a8-923b-0c882f404a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:27.500291 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:27.516472 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-dbda9566-4af1-41a8-923b-0c882f404a07 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:27.517749 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-dbda9566-4af1-41a8-923b-0c882f404a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:27.602188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 231/457] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:27.625808 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227bde0-9c54-4382-9608-7c46d4d21946 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:27.630481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:27.738992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c53f64-0e95-478d-ba70-4cc1a56f3630 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:27.739622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c53f64-0e95-478d-ba70-4cc1a56f3630 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:27.740151 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c53f64-0e95-478d-ba70-4cc1a56f3630 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:27.741012 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227bde0-9c54-4382-9608-7c46d4d21946 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:27.742273 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 232/458] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:26:27.747591 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2cba1-c563-4319-85f8-7ef1d756fee9 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:27.748200 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2cba1-c563-4319-85f8-7ef1d756fee9 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:27.748675 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2cba1-c563-4319-85f8-7ef1d756fee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:27.780830 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7ad2cba1-c563-4319-85f8-7ef1d756fee9 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:27.782452 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7ad2cba1-c563-4319-85f8-7ef1d756fee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:27.796815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598027d0-d3f3-4463-9d8d-142faf096f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:27.798018 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598027d0-d3f3-4463-9d8d-142faf096f32 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:27.798620 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598027d0-d3f3-4463-9d8d-142faf096f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:27.799096 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598027d0-d3f3-4463-9d8d-142faf096f32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:27.812787 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598027d0-d3f3-4463-9d8d-142faf096f32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:27.814717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598027d0-d3f3-4463-9d8d-142faf096f32 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:27.833911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-598027d0-d3f3-4463-9d8d-142faf096f32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:27.833911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:27.833911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:27.833911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:27.833911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:27.833911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:27.833911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:27.833911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:27.833911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:27.833911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:27.833911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:27.833911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:27.833911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:27.833911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:27.833911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:27.834812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:27.834812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:27.834812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:27.834812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:27.834812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:27.834812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:27.834812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:27.834812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:27.834812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:27.834812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:27.834812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:27.834812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:27.834812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:27.834812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:27.834812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:27.834812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:27.834812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:27.835639 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:27.835639 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:27.835639 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:27.835639 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:27.837963 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 227/459] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:27.839378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7a56d-4796-4915-90b4-cb241794648d None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:27.839951 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7a56d-4796-4915-90b4-cb241794648d None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:27.840431 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7a56d-4796-4915-90b4-cb241794648d None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:27.842115 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 228/460] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:27.847534 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7a56d-4796-4915-90b4-cb241794648d None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:27.854662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 233/461] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:26:27.855984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e53d11-2e11-405d-b9ef-e7575b50f5f8 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:27.856269 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e53d11-2e11-405d-b9ef-e7575b50f5f8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:27.856546 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e53d11-2e11-405d-b9ef-e7575b50f5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:27.879412 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-a7e53d11-2e11-405d-b9ef-e7575b50f5f8 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:27.880462 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a7e53d11-2e11-405d-b9ef-e7575b50f5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:27.917894 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:27.951441 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:27.961894 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 234/462] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:27.963147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e53d11-2e11-405d-b9ef-e7575b50f5f8 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:27.986801 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7a56d-4796-4915-90b4-cb241794648d None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:28.032280 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb235b-0dd4-4649-b5aa-30e6cac2e163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:28.033266 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb235b-0dd4-4649-b5aa-30e6cac2e163 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:28.033829 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb235b-0dd4-4649-b5aa-30e6cac2e163 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:28.034492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb235b-0dd4-4649-b5aa-30e6cac2e163 None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:28.035339 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb235b-0dd4-4649-b5aa-30e6cac2e163 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:28.035706 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb235b-0dd4-4649-b5aa-30e6cac2e163 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:28.037388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb235b-0dd4-4649-b5aa-30e6cac2e163 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:28.051984 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 229/463] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:28.053418 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb235b-0dd4-4649-b5aa-30e6cac2e163 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:28.090492 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e53d11-2e11-405d-b9ef-e7575b50f5f8 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:28.128858 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db690be0-b42e-46da-aa3a-8f46302f4310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:28.129632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db690be0-b42e-46da-aa3a-8f46302f4310 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:28.130204 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db690be0-b42e-46da-aa3a-8f46302f4310 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:28.130462 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db690be0-b42e-46da-aa3a-8f46302f4310 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:28.141296 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db690be0-b42e-46da-aa3a-8f46302f4310 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:28.142976 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db690be0-b42e-46da-aa3a-8f46302f4310 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:28.162189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-db690be0-b42e-46da-aa3a-8f46302f4310 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:28.162189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:28.162189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:28.162189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:28.162189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:28.162189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:28.162189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:28.162189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:28.162189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:28.162189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:28.162189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:28.162189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:28.162189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:28.162189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:28.162189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:28.163089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:28.163089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:28.163089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:28.163089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:28.163089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:28.163089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:28.163089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:28.163089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:28.163089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:28.163089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:28.163089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:28.163089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:28.163089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:28.163089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:28.163089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:28.163089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:28.163089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:28.164170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:28.164170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:28.164170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:28.164170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:28.164170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 235/464] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:28.165239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db690be0-b42e-46da-aa3a-8f46302f4310 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:28.188435 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb235b-0dd4-4649-b5aa-30e6cac2e163 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:28.207947 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:28.228722 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:28.229697 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e36a3f-e504-40be-9d38-34c0296b6b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:28.230563 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e36a3f-e504-40be-9d38-34c0296b6b96 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:28.230731 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e36a3f-e504-40be-9d38-34c0296b6b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:28.231111 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e36a3f-e504-40be-9d38-34c0296b6b96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:28.244107 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e36a3f-e504-40be-9d38-34c0296b6b96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:28.246123 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e36a3f-e504-40be-9d38-34c0296b6b96 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:28.263550 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-05e36a3f-e504-40be-9d38-34c0296b6b96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:28.263550 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:28.263550 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:28.263550 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:28.263550 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:28.263550 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:28.263550 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:28.263550 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:28.263550 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:28.263550 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:28.263550 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:28.263550 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:28.263550 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:28.263550 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:28.263550 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:28.264447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:28.264447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:28.264447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:28.264447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:28.264447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:28.264447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:28.264447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:28.264447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:28.264447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:28.264447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:28.264447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:28.264447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:28.264447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:28.264447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:28.264447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:28.264447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:28.264447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:28.265264 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:28.265264 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:28.265264 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:28.265264 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:28.269483 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 230/465] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:28.270408 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e36a3f-e504-40be-9d38-34c0296b6b96 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:28.295025 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db690be0-b42e-46da-aa3a-8f46302f4310 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:28.331557 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47942431-b155-47aa-bd5d-29d94482dc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:28.332182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47942431-b155-47aa-bd5d-29d94482dc91 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:28.332467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47942431-b155-47aa-bd5d-29d94482dc91 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:28.332740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47942431-b155-47aa-bd5d-29d94482dc91 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:28.333429 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47942431-b155-47aa-bd5d-29d94482dc91 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:28.334940 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47942431-b155-47aa-bd5d-29d94482dc91 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:28.361709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 236/466] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Aug 4 04:26:28 2022] POST /identity/v3/users => generated 246 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:26:28.361709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47942431-b155-47aa-bd5d-29d94482dc91 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:28.395243 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e36a3f-e504-40be-9d38-34c0296b6b96 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:28.448225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1888a261-4753-425a-a1ba-5cc480e3c5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:28.448910 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888a261-4753-425a-a1ba-5cc480e3c5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:28.449299 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888a261-4753-425a-a1ba-5cc480e3c5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:28.450034 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888a261-4753-425a-a1ba-5cc480e3c5d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:28.450846 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1888a261-4753-425a-a1ba-5cc480e3c5d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:28.451275 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1888a261-4753-425a-a1ba-5cc480e3c5d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:28.452796 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1888a261-4753-425a-a1ba-5cc480e3c5d0 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:28.467073 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 231/467] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:28.468627 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964ba26-5491-40e2-84e7-9b8c055e8166 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:28.469017 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964ba26-5491-40e2-84e7-9b8c055e8166 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:28.469434 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964ba26-5491-40e2-84e7-9b8c055e8166 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:28.476172 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 232/468] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:28.477195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964ba26-5491-40e2-84e7-9b8c055e8166 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:28.512943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47942431-b155-47aa-bd5d-29d94482dc91 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:28.516265 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:28.529866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:28.558812 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c211027-b5b2-4f80-b1ce-dd68f013e034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:28.559748 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c211027-b5b2-4f80-b1ce-dd68f013e034 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:28.560032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c211027-b5b2-4f80-b1ce-dd68f013e034 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:28.560298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c211027-b5b2-4f80-b1ce-dd68f013e034 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:28.561006 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c211027-b5b2-4f80-b1ce-dd68f013e034 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:28.561313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c211027-b5b2-4f80-b1ce-dd68f013e034 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:28.562803 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c211027-b5b2-4f80-b1ce-dd68f013e034 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:28.578403 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 237/469] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:28.579789 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad352d-d92c-4881-8b7b-a276b37f8c82 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:28.580165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad352d-d92c-4881-8b7b-a276b37f8c82 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:28.580358 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad352d-d92c-4881-8b7b-a276b37f8c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:28.600184 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-fbad352d-d92c-4881-8b7b-a276b37f8c82 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:28.601383 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fbad352d-d92c-4881-8b7b-a276b37f8c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:28.616144 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964ba26-5491-40e2-84e7-9b8c055e8166 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:28.658919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fcedae-3c7e-4e71-a495-330f6dabb1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:28.659952 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcedae-3c7e-4e71-a495-330f6dabb1ba None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:28.660464 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcedae-3c7e-4e71-a495-330f6dabb1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:28.660714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcedae-3c7e-4e71-a495-330f6dabb1ba None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:28.661397 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fcedae-3c7e-4e71-a495-330f6dabb1ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:28.662864 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fcedae-3c7e-4e71-a495-330f6dabb1ba None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:28.687892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 238/470] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:28.688080 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 233/471] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Aug 4 04:26:28 2022] POST /identity/v3/users => generated 248 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:26:28.688967 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad352d-d92c-4881-8b7b-a276b37f8c82 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:28.718673 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fcedae-3c7e-4e71-a495-330f6dabb1ba None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:28.837951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:28.846439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad352d-d92c-4881-8b7b-a276b37f8c82 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:28.857669 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:28.892145 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fcedae-3c7e-4e71-a495-330f6dabb1ba None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:28.914014 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711eef09-097c-4ae3-8628-5a83c9f9bf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:28.914890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711eef09-097c-4ae3-8628-5a83c9f9bf01 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:28.915254 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711eef09-097c-4ae3-8628-5a83c9f9bf01 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:28.915606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711eef09-097c-4ae3-8628-5a83c9f9bf01 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:28.916370 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711eef09-097c-4ae3-8628-5a83c9f9bf01 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:28.917932 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711eef09-097c-4ae3-8628-5a83c9f9bf01 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:28.958406 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 239/472] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Aug 4 04:26:28 2022] POST /identity/v3/users => generated 248 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:26:28.959846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00989c75-a7fa-4d4d-a253-17bf7e84428b None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:28.960215 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00989c75-a7fa-4d4d-a253-17bf7e84428b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:28.960563 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00989c75-a7fa-4d4d-a253-17bf7e84428b None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:28.962663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 240/473] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:28.962796 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2c40d-c8e3-4e4a-8fd0-41f03984ac35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:28.963221 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2c40d-c8e3-4e4a-8fd0-41f03984ac35 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:28.963579 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2c40d-c8e3-4e4a-8fd0-41f03984ac35 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:28.963859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2c40d-c8e3-4e4a-8fd0-41f03984ac35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:28.969343 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cd6c0-3da3-451d-b8c1-308e31a5e0a8 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:28.969689 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cd6c0-3da3-451d-b8c1-308e31a5e0a8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:28.969924 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cd6c0-3da3-451d-b8c1-308e31a5e0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:28.985769 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb2c40d-c8e3-4e4a-8fd0-41f03984ac35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:28.989559 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb2c40d-c8e3-4e4a-8fd0-41f03984ac35 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:28.999687 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-860cd6c0-3da3-451d-b8c1-308e31a5e0a8 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:29.001223 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-860cd6c0-3da3-451d-b8c1-308e31a5e0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:29.020938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-8eb2c40d-c8e3-4e4a-8fd0-41f03984ac35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:29.020938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:29.020938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:29.020938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:29.020938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:29.020938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:29.020938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:29.020938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:29.020938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:29.020938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:29.020938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:29.020938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:29.020938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:29.020938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:29.020938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:29.021931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:29.021931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:29.021931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:29.021931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:29.021931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:29.021931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:29.021931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:29.021931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:29.021931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:29.021931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:29.021931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:29.021931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:29.021931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:29.021931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:29.021931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:29.021931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:29.021931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:29.022774 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:29.022774 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:29.022774 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:29.022774 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:29.030834 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 234/474] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:29.036661 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2c40d-c8e3-4e4a-8fd0-41f03984ac35 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:29.113652 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 241/475] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:29.134864 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cd6c0-3da3-451d-b8c1-308e31a5e0a8 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:29.249014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2c40d-c8e3-4e4a-8fd0-41f03984ac35 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:29.252204 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:29.278853 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cd6c0-3da3-451d-b8c1-308e31a5e0a8 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:29.305702 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c698e31-4f16-4494-a895-a6634d0a2e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:29.306545 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c698e31-4f16-4494-a895-a6634d0a2e75 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:29.306924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c698e31-4f16-4494-a895-a6634d0a2e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:29.307301 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c698e31-4f16-4494-a895-a6634d0a2e75 None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:29.308213 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c698e31-4f16-4494-a895-a6634d0a2e75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:29.308662 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c698e31-4f16-4494-a895-a6634d0a2e75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:29.311995 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c698e31-4f16-4494-a895-a6634d0a2e75 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:29.318734 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:29.328802 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35a418-2e26-4f40-bf99-26d6c73d6b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:29.329723 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35a418-2e26-4f40-bf99-26d6c73d6b64 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:29.330140 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35a418-2e26-4f40-bf99-26d6c73d6b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:29.330578 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35a418-2e26-4f40-bf99-26d6c73d6b64 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:29.332705 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f35a418-2e26-4f40-bf99-26d6c73d6b64 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:29.334626 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f35a418-2e26-4f40-bf99-26d6c73d6b64 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:29.337763 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 235/476] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:29.343584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c698e31-4f16-4494-a895-a6634d0a2e75 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:29.345252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.sql.core [None req-6f35a418-2e26-4f40-bf99-26d6c73d6b64 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 04:26:29.345252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 04 04:26:29.345252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [parameters: {'id': '9a16a0f0c8b740d0a3e2d8e126f7f6ab', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 04 04:26:29.345252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80402) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 04:26:29.367847 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-6f35a418-2e26-4f40-bf99-26d6c73d6b64 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 04 04:26:29.367847 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:29.367847 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 04:26:29.367847 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 04:26:29.367847 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 04:26:29.367847 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 04:26:29.367847 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 04:26:29.367847 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 04:26:29.367847 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 04:26:29.367847 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application conn.query(q) Aug 04 04:26:29.367847 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 04:26:29.367847 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 04:26:29.367847 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 04:26:29.367847 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application result.read() Aug 04 04:26:29.367847 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 04:26:29.368794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 04:26:29.368794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 04:26:29.368794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 04:26:29.368794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 04:26:29.368794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 04:26:29.368794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 04:26:29.368794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 04:26:29.368794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 04:26:29.368794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:29.368794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 04:26:29.368794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:29.368794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:29.368794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 04:26:29.368794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 04:26:29.368794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 04 04:26:29.368794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return ref.to_dict() Aug 04 04:26:29.368794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 04:26:29.368794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application next(self.gen) Aug 04 04:26:29.369757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 04:26:29.369757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application yield resource Aug 04 04:26:29.369757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 04:26:29.369757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application next(self.gen) Aug 04 04:26:29.369757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 04:26:29.369757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 04:26:29.369757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 04:26:29.369757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 04:26:29.369757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 04:26:29.369757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise self.value Aug 04 04:26:29.369757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 04:26:29.369757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 04:26:29.369757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 04:26:29.369757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application session.commit() Aug 04 04:26:29.369757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 04:26:29.369757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 04:26:29.369757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 04:26:29.370589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 04:26:29.370589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 04:26:29.370589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self.session.flush() Aug 04 04:26:29.370589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 04:26:29.370589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 04:26:29.370589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 04:26:29.370589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 04:26:29.370589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 04:26:29.370589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application compat.raise_( Aug 04 04:26:29.370589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 04:26:29.370589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception Aug 04 04:26:29.370589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 04:26:29.370589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 04:26:29.370589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 04:26:29.370589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 04:26:29.370589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 04:26:29.370589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 04:26:29.371476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 04:26:29.371476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 04:26:29.371476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 04 04:26:29.371476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 04 04:26:29.371476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 04:26:29.371476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 04:26:29.371476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 04:26:29.371476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 04:26:29.371476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 04:26:29.371476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 04:26:29.371476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 04:26:29.371476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 04:26:29.371476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 04:26:29.371476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 04:26:29.371476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 04:26:29.371476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception Aug 04 04:26:29.377662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 04:26:29.377662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 04:26:29.377662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 04:26:29.377662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 04:26:29.377662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 04:26:29.377662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 04:26:29.377662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 04:26:29.377662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application conn.query(q) Aug 04 04:26:29.377662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 04:26:29.377662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 04:26:29.377662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 04:26:29.377662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application result.read() Aug 04 04:26:29.377662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 04:26:29.377662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 04:26:29.377662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 04:26:29.377662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 04:26:29.380139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 04:26:29.380139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 04:26:29.380139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 04:26:29.380139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 04:26:29.380139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 04:26:29.380139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 04 04:26:29.380139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [parameters: {'id': '9a16a0f0c8b740d0a3e2d8e126f7f6ab', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 04 04:26:29.380139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 04:26:29.380139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:29.380139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:26:29.380139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:29.380139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:29.380139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:29.380139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:29.380139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:29.380139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:29.381945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:29.381945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:29.381945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:29.381945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:29.381945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:29.381945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:29.381945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 04 04:26:29.381945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 04 04:26:29.381945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:29.381945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:29.381945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 04 04:26:29.381945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 04 04:26:29.381945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 04:26:29.381945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 04:26:29.381945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 04 04:26:29.381945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:29.384193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 242/477] 10.222.0.47 () {60 vars in 1131 bytes} [Thu Aug 4 04:26:29 2022] POST /identity/v3/roles => generated 144 bytes in 237 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:26:29.385739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35a418-2e26-4f40-bf99-26d6c73d6b64 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:29.516184 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c698e31-4f16-4494-a895-a6634d0a2e75 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:29.538027 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35a418-2e26-4f40-bf99-26d6c73d6b64 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:29.556020 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7929c37-0b70-4ae1-a6ba-890fc35bd1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:29.556780 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7929c37-0b70-4ae1-a6ba-890fc35bd1f2 None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:29.557275 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7929c37-0b70-4ae1-a6ba-890fc35bd1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:29.557565 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7929c37-0b70-4ae1-a6ba-890fc35bd1f2 None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:29.558376 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7929c37-0b70-4ae1-a6ba-890fc35bd1f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:29.559908 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7929c37-0b70-4ae1-a6ba-890fc35bd1f2 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:29.587646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 236/478] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Aug 4 04:26:29 2022] POST /identity/v3/users => generated 249 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:26:29.606682 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313cd6c-92ca-40d0-ae27-72e7990c932c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:29.607371 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313cd6c-92ca-40d0-ae27-72e7990c932c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:29.607708 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313cd6c-92ca-40d0-ae27-72e7990c932c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:29.607987 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313cd6c-92ca-40d0-ae27-72e7990c932c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:29.630343 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313cd6c-92ca-40d0-ae27-72e7990c932c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:29.635589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-6313cd6c-92ca-40d0-ae27-72e7990c932c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:26:29.635589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:29.635589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:29.635589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:29.635589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:29.635589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:29.635589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:29.635589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:29.635589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:29.635589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:29.635589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:29.635589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:29.635589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:29.635589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:29.635589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:29.635589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:29.636702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:29.636702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:29.636702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:29.636702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:29.636702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:29.636702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:29.636702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:29.636702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:29.636702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:29.636702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:29.636702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:29.636702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:29.636702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:29.636702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:29.636702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:29.636702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:29.637793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:29.637793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:29.637793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:29.637793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:29.637793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:29.637793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:29.637793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:29.637793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:29.637793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:29.637793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:29.637793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 04:26:29.637793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:29.637793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 243/479] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Aug 4 04:26:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:26:29.645306 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7929c37-0b70-4ae1-a6ba-890fc35bd1f2 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:29.734163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:29.810959 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7929c37-0b70-4ae1-a6ba-890fc35bd1f2 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:29.852778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f030af6-0c42-4205-a9f0-c19cc64f9600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:29.853400 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f030af6-0c42-4205-a9f0-c19cc64f9600 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:29.853704 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f030af6-0c42-4205-a9f0-c19cc64f9600 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:29.853977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f030af6-0c42-4205-a9f0-c19cc64f9600 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:29.854700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f030af6-0c42-4205-a9f0-c19cc64f9600 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:29.854999 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f030af6-0c42-4205-a9f0-c19cc64f9600 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:29.856349 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f030af6-0c42-4205-a9f0-c19cc64f9600 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:29.865912 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 237/480] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:30.879082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658ae0d-7199-449c-a01e-caa2da62bbfd None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:30.879629 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658ae0d-7199-449c-a01e-caa2da62bbfd None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:30.879629 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658ae0d-7199-449c-a01e-caa2da62bbfd None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:30.881258 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 244/481] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:30.895558 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ff689-04bd-48c9-9e6d-5d180743597e None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:30.896089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ff689-04bd-48c9-9e6d-5d180743597e None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:30.896089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ff689-04bd-48c9-9e6d-5d180743597e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:30.922592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-e56ff689-04bd-48c9-9e6d-5d180743597e None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:30.923786 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-e56ff689-04bd-48c9-9e6d-5d180743597e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:31.007797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 238/482] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:31.055643 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658ae0d-7199-449c-a01e-caa2da62bbfd None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:31.063538 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:31.203308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658ae0d-7199-449c-a01e-caa2da62bbfd None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:31.246258 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9226ed5-bfe6-4210-8964-102c81429eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:31.246913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9226ed5-bfe6-4210-8964-102c81429eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:31.247195 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9226ed5-bfe6-4210-8964-102c81429eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:31.247468 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9226ed5-bfe6-4210-8964-102c81429eb5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:31.265686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9226ed5-bfe6-4210-8964-102c81429eb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:31.271918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-c9226ed5-bfe6-4210-8964-102c81429eb5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:31.271918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:31.271918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:31.271918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:31.271918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:31.271918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:31.271918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:31.271918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:31.271918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:31.271918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:31.271918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:31.271918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:31.271918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:31.271918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:31.271918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:31.271918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:31.272778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:31.272778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:31.272778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:31.272778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:31.272778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:31.272778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:31.272778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:31.272778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:31.272778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:31.272778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:31.272778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:31.272778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:31.272778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:31.272778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:31.272778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:31.272778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:31.274068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:31.274068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:31.274068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:31.274068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:31.274068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:31.274068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:31.274068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:31.274068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:31.274068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:31.274068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:31.274068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:31.274068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:31.274068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 245/483] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Aug 4 04:26:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:31.282643 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ff689-04bd-48c9-9e6d-5d180743597e None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:31.350590 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:31.409937 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ff689-04bd-48c9-9e6d-5d180743597e None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:31.446155 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a641c7-4ddf-4338-8cce-9f992913c55e None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:31.446372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a641c7-4ddf-4338-8cce-9f992913c55e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:31.446686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a641c7-4ddf-4338-8cce-9f992913c55e None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:31.448197 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 246/484] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:31.449130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e4aba-b301-4753-ac84-2f1980396b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:31.449772 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e4aba-b301-4753-ac84-2f1980396b2d None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:31.450051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e4aba-b301-4753-ac84-2f1980396b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:31.450320 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e4aba-b301-4753-ac84-2f1980396b2d None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:31.451040 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e4aba-b301-4753-ac84-2f1980396b2d None admin] RBAC: Adding query filter params (name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:31.451337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e4aba-b301-4753-ac84-2f1980396b2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:31.452689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e4aba-b301-4753-ac84-2f1980396b2d None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:31.454804 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ebd99-4dfe-4125-b3dc-ceb290f10138 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:31.455036 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ebd99-4dfe-4125-b3dc-ceb290f10138 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:31.455311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ebd99-4dfe-4125-b3dc-ceb290f10138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:31.464949 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 239/485] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Aug 4 04:26:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:31.470355 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e4aba-b301-4753-ac84-2f1980396b2d None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:31.475727 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-793ebd99-4dfe-4125-b3dc-ceb290f10138 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:31.477153 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-793ebd99-4dfe-4125-b3dc-ceb290f10138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:31.557224 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:31.568755 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 247/486] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:26:31.595305 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ebd99-4dfe-4125-b3dc-ceb290f10138 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:31.647007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e4aba-b301-4753-ac84-2f1980396b2d None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:31.654746 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:31.715420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c41cd-c7c0-4637-95c0-2f5255b56765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:31.717225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c41cd-c7c0-4637-95c0-2f5255b56765 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:31.718303 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c41cd-c7c0-4637-95c0-2f5255b56765 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:31.720484 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ebd99-4dfe-4125-b3dc-ceb290f10138 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:31.720939 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c41cd-c7c0-4637-95c0-2f5255b56765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:31.742062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c41cd-c7c0-4637-95c0-2f5255b56765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:31.743657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c41cd-c7c0-4637-95c0-2f5255b56765 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:31.765628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-467c41cd-c7c0-4637-95c0-2f5255b56765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:31.765628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:31.765628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:31.765628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:31.765628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:31.765628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:31.765628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:31.765628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:31.765628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:31.765628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:31.765628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:31.765628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:31.765628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:31.765628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:31.765628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:31.766502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:31.766502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:31.766502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:31.766502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:31.766502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:31.766502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:31.766502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:31.766502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:31.766502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:31.766502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:31.766502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:31.766502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:31.766502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:31.766502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:31.766502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:31.766502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:31.766502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:31.767927 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:31.767927 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:31.767927 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:31.767927 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:31.767927 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 240/487] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:26:31.768746 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713bd45-a3af-4ef3-b73b-760ec8ac6c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:31.769772 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713bd45-a3af-4ef3-b73b-760ec8ac6c8d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:31.770262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713bd45-a3af-4ef3-b73b-760ec8ac6c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:31.770747 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713bd45-a3af-4ef3-b73b-760ec8ac6c8d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:31.773572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c41cd-c7c0-4637-95c0-2f5255b56765 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:31.785725 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5713bd45-a3af-4ef3-b73b-760ec8ac6c8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:31.796196 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-5713bd45-a3af-4ef3-b73b-760ec8ac6c8d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:31.796196 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:31.796196 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:31.796196 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:31.796196 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:31.796196 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:31.796196 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:31.796196 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:31.796196 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:31.796196 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:31.796196 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:31.796196 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:31.796196 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:31.796196 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:31.796196 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:31.796196 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:31.797097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:31.797097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:31.797097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:31.797097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:31.797097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:31.797097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:31.797097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:31.797097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:31.797097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:31.797097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:31.797097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:31.797097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:31.797097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:31.797097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:31.797097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:31.797097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:31.797916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:31.797916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:31.797916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:31.797916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:31.797916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:31.797916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:31.797916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:31.797916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:31.797916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:31.797916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:31.797916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:31.797916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:31.802868 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 248/488] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Aug 4 04:26:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:31.809755 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713bd45-a3af-4ef3-b73b-760ec8ac6c8d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:31.892433 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:31.922371 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c41cd-c7c0-4637-95c0-2f5255b56765 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:31.941718 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713bd45-a3af-4ef3-b73b-760ec8ac6c8d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:31.961137 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb1858-33e6-49f3-a691-13188dfdd5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:31.962149 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb1858-33e6-49f3-a691-13188dfdd5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:31.962636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb1858-33e6-49f3-a691-13188dfdd5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:31.963136 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb1858-33e6-49f3-a691-13188dfdd5d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:31.964029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb1858-33e6-49f3-a691-13188dfdd5d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:31.964544 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb1858-33e6-49f3-a691-13188dfdd5d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:31.966204 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb1858-33e6-49f3-a691-13188dfdd5d4 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:31.978632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e11dd-2603-43a5-916e-fc480854f777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:31.979478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e11dd-2603-43a5-916e-fc480854f777 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:31.979902 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e11dd-2603-43a5-916e-fc480854f777 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:31.980347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e11dd-2603-43a5-916e-fc480854f777 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:31.981212 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e11dd-2603-43a5-916e-fc480854f777 None admin] RBAC: Adding query filter params (name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:31.981337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 241/489] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:31.981861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e11dd-2603-43a5-916e-fc480854f777 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:31.982858 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75c326-4037-41db-839e-37e6d2a15b92 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:31.983215 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75c326-4037-41db-839e-37e6d2a15b92 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:31.983560 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75c326-4037-41db-839e-37e6d2a15b92 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:31.984003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e11dd-2603-43a5-916e-fc480854f777 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:31.984937 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 242/490] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:31.986235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75c326-4037-41db-839e-37e6d2a15b92 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:31.998606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 249/491] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Aug 4 04:26:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:31.999664 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470c2cb-222f-4ea5-a88e-287c92447ab8 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:31.999954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470c2cb-222f-4ea5-a88e-287c92447ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:32.000230 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470c2cb-222f-4ea5-a88e-287c92447ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:32.009326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:32.018962 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-a470c2cb-222f-4ea5-a88e-287c92447ab8 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:32.020228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a470c2cb-222f-4ea5-a88e-287c92447ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:32.098858 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 250/492] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:32.099909 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470c2cb-222f-4ea5-a88e-287c92447ab8 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:32.103799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75c326-4037-41db-839e-37e6d2a15b92 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:32.148456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa162f-dd44-492b-84e0-e3070ce6dd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:32.149480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa162f-dd44-492b-84e0-e3070ce6dd83 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:32.149983 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa162f-dd44-492b-84e0-e3070ce6dd83 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:32.150473 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa162f-dd44-492b-84e0-e3070ce6dd83 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:32.158290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa162f-dd44-492b-84e0-e3070ce6dd83 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:32.160527 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa162f-dd44-492b-84e0-e3070ce6dd83 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:32.163318 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:32.176014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-0baa162f-dd44-492b-84e0-e3070ce6dd83 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 04:26:32.176014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:32.176014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:32.176014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:32.176014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:32.176014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:32.176014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:32.176014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:32.176014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:32.176014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:32.176014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:32.176014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:32.176014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:26:32.176014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:26:32.176014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:26:32.176014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:26:32.176968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:32.176968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:32.176968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:26:32.176968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:32.176968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:26:32.176968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:32.176968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:32.176968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:32.176968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:32.176968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:32.176968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:32.176968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:32.176968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:32.176968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:32.176968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:32.176968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:32.176968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:32.176968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:32.177926 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:32.177926 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:32.177926 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:26:32.177926 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:26:32.177926 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:26:32.177926 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:26:32.177926 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:26:32.177926 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:26:32.177926 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 04:26:32.177926 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:32.180572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 243/493] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Aug 4 04:26:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:26:32.181702 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa162f-dd44-492b-84e0-e3070ce6dd83 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:32.222077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470c2cb-222f-4ea5-a88e-287c92447ab8 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:32.258113 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adc2fc-90aa-4cc8-ba86-fde531d80023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:32.258808 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adc2fc-90aa-4cc8-ba86-fde531d80023 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:32.259131 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adc2fc-90aa-4cc8-ba86-fde531d80023 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:32.260149 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adc2fc-90aa-4cc8-ba86-fde531d80023 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:32.269512 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:32.271437 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44adc2fc-90aa-4cc8-ba86-fde531d80023 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:32.272928 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44adc2fc-90aa-4cc8-ba86-fde531d80023 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:32.289390 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-44adc2fc-90aa-4cc8-ba86-fde531d80023 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:32.289390 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:32.289390 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:32.289390 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:32.289390 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:32.289390 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:32.289390 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:32.289390 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:32.289390 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:32.289390 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:32.289390 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:32.289390 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:32.289390 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:32.289390 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:32.289390 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:32.290336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:32.290336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:32.290336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:32.290336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:32.290336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:32.290336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:32.290336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:32.290336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:32.290336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:32.290336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:32.290336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:32.290336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:32.290336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:32.290336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:32.290336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:32.290336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:32.290336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:32.291792 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:32.291792 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:32.291792 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:32.291792 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:32.291792 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 251/494] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:32.291792 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adc2fc-90aa-4cc8-ba86-fde531d80023 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:32.298593 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa162f-dd44-492b-84e0-e3070ce6dd83 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:32.336070 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9a894-a31f-495d-b27b-16c3f93135e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:32.336731 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9a894-a31f-495d-b27b-16c3f93135e7 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:32.337013 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9a894-a31f-495d-b27b-16c3f93135e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:32.337291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9a894-a31f-495d-b27b-16c3f93135e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:32.348208 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9a894-a31f-495d-b27b-16c3f93135e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:32.353892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-16d9a894-a31f-495d-b27b-16c3f93135e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:32.353892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:32.353892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:32.353892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:32.353892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:32.353892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:32.353892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:32.353892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:32.353892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:32.353892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:32.353892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:32.353892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:32.353892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:32.353892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:32.353892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:32.353892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:32.354782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:32.354782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:32.354782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:32.354782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:32.354782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:32.354782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:32.354782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:32.354782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:32.354782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:32.354782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:32.354782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:32.354782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:32.354782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:32.354782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:32.354782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:32.354782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:32.355561 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:32.355561 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:32.355561 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:32.355561 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:32.355561 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:32.355561 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:32.355561 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:32.355561 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:32.355561 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:32.355561 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:32.355561 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:32.355561 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:32.355561 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 244/495] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Aug 4 04:26:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:32.356208 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9a894-a31f-495d-b27b-16c3f93135e7 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:32.405273 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:32.410061 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adc2fc-90aa-4cc8-ba86-fde531d80023 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:32.450014 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7687c905-6e20-4a2a-9770-f179c753d575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:32.450635 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687c905-6e20-4a2a-9770-f179c753d575 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:32.450915 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687c905-6e20-4a2a-9770-f179c753d575 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:32.451186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687c905-6e20-4a2a-9770-f179c753d575 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:32.451988 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7687c905-6e20-4a2a-9770-f179c753d575 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:32.452299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7687c905-6e20-4a2a-9770-f179c753d575 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:32.453735 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7687c905-6e20-4a2a-9770-f179c753d575 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:32.465171 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9a894-a31f-495d-b27b-16c3f93135e7 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:32.492539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 252/496] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Aug 4 04:26:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:32.493738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abddd2-cb89-408c-a3fd-0b53d419be0d None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:32.494030 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abddd2-cb89-408c-a3fd-0b53d419be0d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:32.494311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abddd2-cb89-408c-a3fd-0b53d419be0d None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:32.495601 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 253/497] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:32.496328 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abddd2-cb89-408c-a3fd-0b53d419be0d None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:32.515399 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27709ccd-0a29-44ce-ac4c-9d5c35593a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:32.516164 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27709ccd-0a29-44ce-ac4c-9d5c35593a4c None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:32.516443 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27709ccd-0a29-44ce-ac4c-9d5c35593a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:32.516710 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27709ccd-0a29-44ce-ac4c-9d5c35593a4c None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:32.517403 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27709ccd-0a29-44ce-ac4c-9d5c35593a4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:32.517732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27709ccd-0a29-44ce-ac4c-9d5c35593a4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:32.519187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27709ccd-0a29-44ce-ac4c-9d5c35593a4c None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:32.526339 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:32.532577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 245/498] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:32.533358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27709ccd-0a29-44ce-ac4c-9d5c35593a4c None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:32.600406 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abddd2-cb89-408c-a3fd-0b53d419be0d None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:32.633102 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27709ccd-0a29-44ce-ac4c-9d5c35593a4c None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:32.634481 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ea2ab-12cd-42c7-9805-39068f0525b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:32.635096 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ea2ab-12cd-42c7-9805-39068f0525b7 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:32.635360 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ea2ab-12cd-42c7-9805-39068f0525b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:32.635632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ea2ab-12cd-42c7-9805-39068f0525b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:32.636343 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ea2ab-12cd-42c7-9805-39068f0525b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:32.636646 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ea2ab-12cd-42c7-9805-39068f0525b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:32.638089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ea2ab-12cd-42c7-9805-39068f0525b7 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:32.646390 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 254/499] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Aug 4 04:26:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:32.647618 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0433080-3a06-49b8-9c69-af5ad66def58 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:32.647909 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0433080-3a06-49b8-9c69-af5ad66def58 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:32.648184 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0433080-3a06-49b8-9c69-af5ad66def58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:32.665212 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-d0433080-3a06-49b8-9c69-af5ad66def58 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:32.666427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d0433080-3a06-49b8-9c69-af5ad66def58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:32.670504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f46447-ebed-4ce2-8071-5ba027c253a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:32.671445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f46447-ebed-4ce2-8071-5ba027c253a0 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:32.671716 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f46447-ebed-4ce2-8071-5ba027c253a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:32.671982 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f46447-ebed-4ce2-8071-5ba027c253a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:32.680228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:32.733489 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 255/500] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:32.746036 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c705c-945e-402c-a1d6-da8c96927846 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:32.746036 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c705c-945e-402c-a1d6-da8c96927846 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:32.746036 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c705c-945e-402c-a1d6-da8c96927846 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:32.746036 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 256/501] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:32.746303 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:32.746373 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c705c-945e-402c-a1d6-da8c96927846 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:32.746439 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f46447-ebed-4ce2-8071-5ba027c253a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:32.746439 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f46447-ebed-4ce2-8071-5ba027c253a0 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:32.845618 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 246/502] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:26:32.847252 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f46447-ebed-4ce2-8071-5ba027c253a0 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:32.847380 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c705c-945e-402c-a1d6-da8c96927846 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:32.882011 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb21dd6-4cf3-4606-a539-217aae8f9fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:32.882730 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb21dd6-4cf3-4606-a539-217aae8f9fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:32.883074 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb21dd6-4cf3-4606-a539-217aae8f9fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:32.883348 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb21dd6-4cf3-4606-a539-217aae8f9fc1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:32.890274 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb21dd6-4cf3-4606-a539-217aae8f9fc1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:32.891725 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb21dd6-4cf3-4606-a539-217aae8f9fc1 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:32.905408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-1fb21dd6-4cf3-4606-a539-217aae8f9fc1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 04:26:32.905408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:32.905408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:32.905408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:32.905408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:32.905408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:32.905408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:32.905408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:32.905408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:32.905408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:32.905408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:32.905408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:32.905408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:26:32.905408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:26:32.905408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:26:32.905408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:26:32.906781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:32.906781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:32.906781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:26:32.906781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:32.906781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:26:32.906781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:32.906781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:32.906781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:32.906781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:32.906781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:32.906781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:32.906781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:32.906781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:32.906781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:32.906781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:32.906781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:32.906781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:32.906781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:32.907642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:32.907642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:32.907642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:26:32.907642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:26:32.907642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:26:32.907642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:26:32.907642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:26:32.907642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:26:32.907642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 04:26:32.907642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:32.907642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 257/503] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:32.908176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396f128-e84b-413f-8982-0be02f35b846 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:32.908176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396f128-e84b-413f-8982-0be02f35b846 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:32.908349 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396f128-e84b-413f-8982-0be02f35b846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:32.927337 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-8396f128-e84b-413f-8982-0be02f35b846 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:32.928518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8396f128-e84b-413f-8982-0be02f35b846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:32.930538 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:32.950919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:32.955057 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f46447-ebed-4ce2-8071-5ba027c253a0 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:32.990587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ade065-b70a-4891-8ace-9c9271ecd441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:32.991223 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ade065-b70a-4891-8ace-9c9271ecd441 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:32.991521 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ade065-b70a-4891-8ace-9c9271ecd441 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:32.991816 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ade065-b70a-4891-8ace-9c9271ecd441 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:32.996790 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 258/504] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:32.997564 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396f128-e84b-413f-8982-0be02f35b846 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:33.004377 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ade065-b70a-4891-8ace-9c9271ecd441 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:33.006288 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ade065-b70a-4891-8ace-9c9271ecd441 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:33.022624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-e9ade065-b70a-4891-8ace-9c9271ecd441 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:33.022624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:33.022624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:33.022624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:33.022624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:33.022624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:33.022624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:33.022624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:33.022624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:33.022624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:33.022624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:33.022624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:33.022624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:33.022624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:33.022624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:33.023739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:33.023739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:33.023739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:33.023739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:33.023739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:33.023739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:33.023739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:33.023739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:33.023739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:33.023739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:33.023739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:33.023739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:33.023739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:33.023739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:33.023739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:33.023739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:33.023739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:33.024589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:33.024589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:33.024589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:33.024589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:33.024589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 247/505] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:33.024845 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ade065-b70a-4891-8ace-9c9271ecd441 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:33.102157 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396f128-e84b-413f-8982-0be02f35b846 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:33.129558 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ade065-b70a-4891-8ace-9c9271ecd441 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:33.135326 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da7abc-47ac-4d30-8b72-0ded5e62d3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:33.135959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da7abc-47ac-4d30-8b72-0ded5e62d3e8 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:33.136246 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da7abc-47ac-4d30-8b72-0ded5e62d3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:33.136537 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da7abc-47ac-4d30-8b72-0ded5e62d3e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:33.143885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:33.147763 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23da7abc-47ac-4d30-8b72-0ded5e62d3e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:33.152970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-23da7abc-47ac-4d30-8b72-0ded5e62d3e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:33.152970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:33.152970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:33.152970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:33.152970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:33.152970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:33.152970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:33.152970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:33.152970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:33.152970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:33.152970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:33.152970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:33.152970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:33.152970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:33.152970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:33.152970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:33.153904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:33.153904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:33.153904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:33.153904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:33.153904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:33.153904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:33.153904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:33.153904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:33.153904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:33.153904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:33.153904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:33.153904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:33.153904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:33.153904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:33.153904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:33.153904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:33.154684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:33.154684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:33.154684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:33.154684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:33.154684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:33.154684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:33.154684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:33.154684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:33.154684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:33.154684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:33.154684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:33.154684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:33.154684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 259/506] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Aug 4 04:26:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:33.155323 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da7abc-47ac-4d30-8b72-0ded5e62d3e8 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:33.164091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae885ef-222f-423f-a6cf-0dfda0395f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:33.164846 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae885ef-222f-423f-a6cf-0dfda0395f86 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:33.165186 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae885ef-222f-423f-a6cf-0dfda0395f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:33.165494 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae885ef-222f-423f-a6cf-0dfda0395f86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:33.176865 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae885ef-222f-423f-a6cf-0dfda0395f86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:33.178328 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae885ef-222f-423f-a6cf-0dfda0395f86 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:33.192621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-1ae885ef-222f-423f-a6cf-0dfda0395f86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:33.192621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:33.192621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:33.192621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:33.192621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:33.192621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:33.192621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:33.192621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:33.192621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:33.192621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:33.192621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:33.192621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:33.192621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:33.192621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:33.192621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:33.193867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:33.193867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:33.193867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:33.193867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:33.193867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:33.193867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:33.193867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:33.193867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:33.193867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:33.193867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:33.193867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:33.193867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:33.193867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:33.193867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:33.193867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:33.193867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:33.193867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:33.194690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:33.194690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:33.194690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:33.194690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:33.194690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 248/507] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:33.194690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae885ef-222f-423f-a6cf-0dfda0395f86 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:33.197572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:33.260292 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da7abc-47ac-4d30-8b72-0ded5e62d3e8 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:33.293849 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53076584-0aea-4c4b-ae19-830bf6d9f541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:33.294462 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53076584-0aea-4c4b-ae19-830bf6d9f541 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:33.294743 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53076584-0aea-4c4b-ae19-830bf6d9f541 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:33.294964 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae885ef-222f-423f-a6cf-0dfda0395f86 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:33.295064 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53076584-0aea-4c4b-ae19-830bf6d9f541 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:33.296087 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53076584-0aea-4c4b-ae19-830bf6d9f541 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:33.296397 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53076584-0aea-4c4b-ae19-830bf6d9f541 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:33.297873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53076584-0aea-4c4b-ae19-830bf6d9f541 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:33.315772 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 260/508] 10.222.0.47 () {56 vars in 1137 bytes} [Thu Aug 4 04:26:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:33.316728 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53076584-0aea-4c4b-ae19-830bf6d9f541 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:33.329507 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56c210-8c75-4e70-8b12-ff285f6ae2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:33.330170 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56c210-8c75-4e70-8b12-ff285f6ae2ff None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:33.330453 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56c210-8c75-4e70-8b12-ff285f6ae2ff None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:33.330783 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56c210-8c75-4e70-8b12-ff285f6ae2ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:33.341245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba56c210-8c75-4e70-8b12-ff285f6ae2ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:33.342725 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba56c210-8c75-4e70-8b12-ff285f6ae2ff None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:33.356729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-ba56c210-8c75-4e70-8b12-ff285f6ae2ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:33.356729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:33.356729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:33.356729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:33.356729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:33.356729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:33.356729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:33.356729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:33.356729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:33.356729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:33.356729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:33.356729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:33.356729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:33.356729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:33.356729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:33.357726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:33.357726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:33.357726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:33.357726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:33.357726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:33.357726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:33.357726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:33.357726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:33.357726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:33.357726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:33.357726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:33.357726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:33.357726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:33.357726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:33.357726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:33.357726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:33.357726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:33.359103 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:33.359103 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:33.359103 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:33.359103 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:33.359103 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 249/509] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:33.359103 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56c210-8c75-4e70-8b12-ff285f6ae2ff None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:33.378578 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:33.416395 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:33.423821 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53076584-0aea-4c4b-ae19-830bf6d9f541 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:33.458967 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf158407-9dca-4852-b8ff-229c58347af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:33.459919 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf158407-9dca-4852-b8ff-229c58347af8 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:33.459919 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf158407-9dca-4852-b8ff-229c58347af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:33.460176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf158407-9dca-4852-b8ff-229c58347af8 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:33.460868 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf158407-9dca-4852-b8ff-229c58347af8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:33.461168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf158407-9dca-4852-b8ff-229c58347af8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:33.462589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf158407-9dca-4852-b8ff-229c58347af8 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:33.463576 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56c210-8c75-4e70-8b12-ff285f6ae2ff None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:33.473878 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 261/510] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:33.474750 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf158407-9dca-4852-b8ff-229c58347af8 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:33.502268 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b42a90-e8f1-4d6a-aac7-44855917ee3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:33.503530 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b42a90-e8f1-4d6a-aac7-44855917ee3a None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:33.503808 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b42a90-e8f1-4d6a-aac7-44855917ee3a None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:33.504081 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b42a90-e8f1-4d6a-aac7-44855917ee3a None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:33.504791 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b42a90-e8f1-4d6a-aac7-44855917ee3a None admin] RBAC: Adding query filter params (name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:33.505092 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b42a90-e8f1-4d6a-aac7-44855917ee3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:33.506477 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b42a90-e8f1-4d6a-aac7-44855917ee3a None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:33.515868 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 250/511] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Aug 4 04:26:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:33.516605 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b42a90-e8f1-4d6a-aac7-44855917ee3a None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:33.580130 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf158407-9dca-4852-b8ff-229c58347af8 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:33.596174 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:33.614881 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19475917-5ea6-4711-a50b-d19b4e6a0b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:33.615497 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19475917-5ea6-4711-a50b-d19b4e6a0b9c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:33.615779 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19475917-5ea6-4711-a50b-d19b4e6a0b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:33.616059 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19475917-5ea6-4711-a50b-d19b4e6a0b9c None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:33.616741 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19475917-5ea6-4711-a50b-d19b4e6a0b9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:33.617037 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19475917-5ea6-4711-a50b-d19b4e6a0b9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:33.618432 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19475917-5ea6-4711-a50b-d19b4e6a0b9c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:33.624772 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b42a90-e8f1-4d6a-aac7-44855917ee3a None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:33.630949 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 262/512] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:33.632786 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19475917-5ea6-4711-a50b-d19b4e6a0b9c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:33.641167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:33.660174 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea54fe-1d2e-403f-bb4d-21f2e10c73f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:33.660786 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea54fe-1d2e-403f-bb4d-21f2e10c73f5 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:33.661067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea54fe-1d2e-403f-bb4d-21f2e10c73f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:33.661348 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea54fe-1d2e-403f-bb4d-21f2e10c73f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:33.731250 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea54fe-1d2e-403f-bb4d-21f2e10c73f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:33.731433 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19475917-5ea6-4711-a50b-d19b4e6a0b9c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:33.732702 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea54fe-1d2e-403f-bb4d-21f2e10c73f5 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:33.765954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99baf40f-583c-439b-899a-49255bd98ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:33.766502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99baf40f-583c-439b-899a-49255bd98ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:33.766785 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99baf40f-583c-439b-899a-49255bd98ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:33.767060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99baf40f-583c-439b-899a-49255bd98ac2 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:33.767744 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99baf40f-583c-439b-899a-49255bd98ac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:33.768048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99baf40f-583c-439b-899a-49255bd98ac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:33.769527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99baf40f-583c-439b-899a-49255bd98ac2 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:33.780877 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 263/513] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:33.781689 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99baf40f-583c-439b-899a-49255bd98ac2 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:33.820673 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:33.830305 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 251/514] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:26:33.832012 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea54fe-1d2e-403f-bb4d-21f2e10c73f5 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:33.849343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:33.889219 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99baf40f-583c-439b-899a-49255bd98ac2 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:33.925042 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c454c-6cd9-4537-a1cd-44a965e6ff26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:33.925930 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c454c-6cd9-4537-a1cd-44a965e6ff26 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:33.926218 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c454c-6cd9-4537-a1cd-44a965e6ff26 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:33.926486 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c454c-6cd9-4537-a1cd-44a965e6ff26 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:33.931752 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea54fe-1d2e-403f-bb4d-21f2e10c73f5 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:33.933959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c454c-6cd9-4537-a1cd-44a965e6ff26 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:33.935426 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c454c-6cd9-4537-a1cd-44a965e6ff26 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:33.947135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-815c454c-6cd9-4537-a1cd-44a965e6ff26 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 04:26:33.947135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:33.947135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:33.947135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:33.947135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:33.947135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:33.947135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:33.947135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:33.947135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:33.947135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:33.947135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:33.947135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:33.947135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:26:33.947135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:26:33.947135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:26:33.947135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:26:33.948165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:33.948165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:33.948165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:26:33.948165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:33.948165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:26:33.948165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:33.948165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:33.948165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:33.948165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:33.948165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:33.948165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:33.948165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:33.948165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:33.948165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:33.948165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:33.948165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:33.948165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:33.948165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:33.949052 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:33.949052 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:33.949052 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:26:33.949052 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:26:33.949052 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:26:33.949052 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:26:33.949052 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:26:33.949052 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:26:33.949052 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 04:26:33.949052 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:33.949052 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 264/515] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:33.949052 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c454c-6cd9-4537-a1cd-44a965e6ff26 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:33.970002 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029494ff-f9e0-458a-959d-1c8f5fb504f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:33.970726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029494ff-f9e0-458a-959d-1c8f5fb504f9 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:33.971006 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029494ff-f9e0-458a-959d-1c8f5fb504f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:33.971305 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029494ff-f9e0-458a-959d-1c8f5fb504f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:33.981811 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029494ff-f9e0-458a-959d-1c8f5fb504f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:33.983291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029494ff-f9e0-458a-959d-1c8f5fb504f9 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:33.998491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-029494ff-f9e0-458a-959d-1c8f5fb504f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:33.998491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:33.998491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:33.998491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:33.998491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:33.998491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:33.998491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:33.998491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:33.998491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:33.998491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:33.998491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:33.998491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:33.998491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:33.998491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:33.998491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:33.999504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:33.999504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:33.999504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:33.999504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:33.999504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:33.999504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:33.999504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:33.999504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:33.999504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:33.999504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:33.999504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:33.999504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:33.999504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:33.999504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:33.999504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:33.999504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:33.999504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:34.000528 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:34.000528 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:34.000528 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:34.000528 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:34.004552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 252/516] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:34.005306 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029494ff-f9e0-458a-959d-1c8f5fb504f9 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:34.042128 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:34.049368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c454c-6cd9-4537-a1cd-44a965e6ff26 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:34.071686 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:34.083135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef3f0ed-4495-41c4-8d79-06c7f0764fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:34.083776 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3f0ed-4495-41c4-8d79-06c7f0764fed None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:34.084063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3f0ed-4495-41c4-8d79-06c7f0764fed None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:34.084339 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3f0ed-4495-41c4-8d79-06c7f0764fed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:34.091035 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef3f0ed-4495-41c4-8d79-06c7f0764fed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:34.092485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef3f0ed-4495-41c4-8d79-06c7f0764fed None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:34.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-4ef3f0ed-4495-41c4-8d79-06c7f0764fed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:34.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:34.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:34.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:34.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:34.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:34.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:34.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:34.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:34.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:34.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:34.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:34.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:26:34.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:26:34.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:26:34.105835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:26:34.105835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:34.105835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:34.105835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:34.105835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:34.105835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:34.105835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:34.105835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:34.105835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:34.105835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:34.105835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:34.105835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:34.105835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:34.105835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:34.105835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:34.105835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:34.105835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:34.106671 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:26:34.106671 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:26:34.106671 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:26:34.106671 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:26:34.106671 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:26:34.106671 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:26:34.106671 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:34.106671 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:34.106671 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 265/517] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:34.107134 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029494ff-f9e0-458a-959d-1c8f5fb504f9 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:34.107203 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef3f0ed-4495-41c4-8d79-06c7f0764fed None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:34.146064 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25471fc-7c09-4815-8765-9027a3544086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:34.146669 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25471fc-7c09-4815-8765-9027a3544086 None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:34.146949 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25471fc-7c09-4815-8765-9027a3544086 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:34.147215 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25471fc-7c09-4815-8765-9027a3544086 None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:34.147888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25471fc-7c09-4815-8765-9027a3544086 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:34.149294 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25471fc-7c09-4815-8765-9027a3544086 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:34.171088 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 253/518] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Aug 4 04:26:34 2022] POST /identity/v3/users => generated 247 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:26:34.171884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25471fc-7c09-4815-8765-9027a3544086 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:34.222932 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef3f0ed-4495-41c4-8d79-06c7f0764fed None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:34.266193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb4369-4693-4c49-8181-d827514a04d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:34.267009 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb4369-4693-4c49-8181-d827514a04d6 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:34.267358 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb4369-4693-4c49-8181-d827514a04d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:34.267640 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb4369-4693-4c49-8181-d827514a04d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:34.280320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbb4369-4693-4c49-8181-d827514a04d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:34.281859 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbb4369-4693-4c49-8181-d827514a04d6 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:34.288671 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25471fc-7c09-4815-8765-9027a3544086 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:34.298969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-7dbb4369-4693-4c49-8181-d827514a04d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:34.298969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:34.298969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:34.298969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:34.298969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:34.298969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:34.298969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:34.298969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:34.298969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:34.298969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:34.298969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:34.298969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:34.298969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:34.298969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:34.298969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:34.300451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:34.300451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:34.300451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:34.300451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:34.300451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:34.300451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:34.300451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:34.300451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:34.300451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:34.300451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:34.300451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:34.300451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:34.300451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:34.300451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:34.300451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:34.300451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:34.300451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:34.301274 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:34.301274 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:34.301274 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:34.301274 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:34.301274 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 266/519] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:34.302101 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb4369-4693-4c49-8181-d827514a04d6 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:34.313726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:34.323944 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:34.338410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3ea07-0867-4384-b8e4-739a7e045f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:34.339417 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3ea07-0867-4384-b8e4-739a7e045f78 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:34.339856 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3ea07-0867-4384-b8e4-739a7e045f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:34.340182 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3ea07-0867-4384-b8e4-739a7e045f78 None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:34.340993 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f3ea07-0867-4384-b8e4-739a7e045f78 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:34.341294 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f3ea07-0867-4384-b8e4-739a7e045f78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:34.342817 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f3ea07-0867-4384-b8e4-739a7e045f78 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:34.382618 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 254/520] 10.222.0.47 () {56 vars in 1137 bytes} [Thu Aug 4 04:26:34 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:34.383547 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3ea07-0867-4384-b8e4-739a7e045f78 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:34.420263 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb4369-4693-4c49-8181-d827514a04d6 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:34.457188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2701fc6-2af2-40a7-b93a-3e7699066dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:34.457933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2701fc6-2af2-40a7-b93a-3e7699066dee None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:34.458283 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2701fc6-2af2-40a7-b93a-3e7699066dee None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:34.458558 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2701fc6-2af2-40a7-b93a-3e7699066dee None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:34.459250 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2701fc6-2af2-40a7-b93a-3e7699066dee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:34.459555 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2701fc6-2af2-40a7-b93a-3e7699066dee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:34.460922 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2701fc6-2af2-40a7-b93a-3e7699066dee None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:34.474541 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 267/521] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:34.475368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2701fc6-2af2-40a7-b93a-3e7699066dee None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:34.512862 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3ea07-0867-4384-b8e4-739a7e045f78 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:34.550675 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:34.551286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723effa2-43db-40cb-ba9a-977ecc9bb727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:34.552075 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723effa2-43db-40cb-ba9a-977ecc9bb727 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:34.552483 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723effa2-43db-40cb-ba9a-977ecc9bb727 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:34.552897 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723effa2-43db-40cb-ba9a-977ecc9bb727 None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:34.553684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723effa2-43db-40cb-ba9a-977ecc9bb727 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:34.553983 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723effa2-43db-40cb-ba9a-977ecc9bb727 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:34.555804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723effa2-43db-40cb-ba9a-977ecc9bb727 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:34.574596 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 255/522] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:26:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:34.575190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723effa2-43db-40cb-ba9a-977ecc9bb727 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:34.592162 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2701fc6-2af2-40a7-b93a-3e7699066dee None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:34.621719 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:34.629219 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a52ce7-6a27-4ef2-9d35-5b365b97be02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:34.629931 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a52ce7-6a27-4ef2-9d35-5b365b97be02 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:34.630210 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a52ce7-6a27-4ef2-9d35-5b365b97be02 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:34.630488 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a52ce7-6a27-4ef2-9d35-5b365b97be02 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:34.631187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a52ce7-6a27-4ef2-9d35-5b365b97be02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:34.631493 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a52ce7-6a27-4ef2-9d35-5b365b97be02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:34.633033 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a52ce7-6a27-4ef2-9d35-5b365b97be02 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:34.650782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 268/523] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:34.651726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a52ce7-6a27-4ef2-9d35-5b365b97be02 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:34.686110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723effa2-43db-40cb-ba9a-977ecc9bb727 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:34.740781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d9fdb-97b8-4915-af27-2d93cbfe73ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:34.741933 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d9fdb-97b8-4915-af27-2d93cbfe73ce None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:34.742357 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d9fdb-97b8-4915-af27-2d93cbfe73ce None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:34.742811 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d9fdb-97b8-4915-af27-2d93cbfe73ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:34.789517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a52ce7-6a27-4ef2-9d35-5b365b97be02 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:34.821006 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:34.830605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a805a-8bf5-48b1-a1d7-ca6152e21838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:34.832640 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a805a-8bf5-48b1-a1d7-ca6152e21838 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:34.832927 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a805a-8bf5-48b1-a1d7-ca6152e21838 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:34.833357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a805a-8bf5-48b1-a1d7-ca6152e21838 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:34.834325 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d9fdb-97b8-4915-af27-2d93cbfe73ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:34.835779 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d9fdb-97b8-4915-af27-2d93cbfe73ce None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:34.842478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a805a-8bf5-48b1-a1d7-ca6152e21838 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:34.844343 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a805a-8bf5-48b1-a1d7-ca6152e21838 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:34.860063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-1f3a805a-8bf5-48b1-a1d7-ca6152e21838 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 04:26:34.860063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:34.860063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:34.860063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:34.860063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:34.860063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:34.860063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:34.860063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:34.860063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:34.860063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:34.860063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:34.860063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:34.860063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:26:34.860063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:26:34.860063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:26:34.860063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:26:34.861182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:34.861182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:34.861182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:26:34.861182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:34.861182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:26:34.861182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:34.861182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:34.861182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:34.861182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:34.861182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:34.861182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:34.861182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:34.861182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:34.861182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:34.861182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:34.861182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:34.861182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:34.861182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:34.862186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:34.862186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:34.862186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:26:34.862186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:26:34.862186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:26:34.862186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:26:34.862186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:26:34.862186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:26:34.862186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 04:26:34.862186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:34.862186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 269/524] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Aug 4 04:26:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:34.862890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a805a-8bf5-48b1-a1d7-ca6152e21838 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:34.888278 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:34.954265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 256/525] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:26:34.955504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d9fdb-97b8-4915-af27-2d93cbfe73ce None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:34.989999 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a805a-8bf5-48b1-a1d7-ca6152e21838 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:35.025602 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ebd8e-94d7-43b0-a6a0-de7a352f48de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:35.026217 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ebd8e-94d7-43b0-a6a0-de7a352f48de None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:35.026500 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ebd8e-94d7-43b0-a6a0-de7a352f48de None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:35.026777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ebd8e-94d7-43b0-a6a0-de7a352f48de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:35.027566 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ebd8e-94d7-43b0-a6a0-de7a352f48de None admin] RBAC: Adding query filter params (role.id=cc86b64693b6417e90ad9361460574b0, scope.project.id=f37a5f3140ec431da8ea14568654114a, user.id=caa2901451aa4554bdfcac1bddbc620a) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:35.027882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ebd8e-94d7-43b0-a6a0-de7a352f48de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:35.029329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ebd8e-94d7-43b0-a6a0-de7a352f48de None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:35.035406 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 270/526] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:26:34 2022] GET /identity/v3/role_assignments?user.id=caa2901451aa4554bdfcac1bddbc620a&scope.project.id=f37a5f3140ec431da8ea14568654114a&role.id=cc86b64693b6417e90ad9361460574b0 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:35.043100 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ebd8e-94d7-43b0-a6a0-de7a352f48de None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:35.068838 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d9fdb-97b8-4915-af27-2d93cbfe73ce None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:35.113189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7db46-3d6b-4f63-9200-f96a5fb0b8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:35.114317 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7db46-3d6b-4f63-9200-f96a5fb0b8fd None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:35.114882 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7db46-3d6b-4f63-9200-f96a5fb0b8fd None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:35.115370 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7db46-3d6b-4f63-9200-f96a5fb0b8fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:35.123285 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:35.126530 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f7db46-3d6b-4f63-9200-f96a5fb0b8fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:35.129249 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f7db46-3d6b-4f63-9200-f96a5fb0b8fd None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:35.147778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-d0f7db46-3d6b-4f63-9200-f96a5fb0b8fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:35.147778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:35.147778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:35.147778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:35.147778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:35.147778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:35.147778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:35.147778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:35.147778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:35.147778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:35.147778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:35.147778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:35.147778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:26:35.147778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:26:35.147778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:26:35.148689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:26:35.148689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:35.148689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:35.148689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:35.148689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:35.148689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:35.148689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:35.148689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:35.148689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:35.148689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:35.148689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:35.148689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:35.148689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:35.148689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:35.148689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:35.148689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:35.148689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:35.149505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:26:35.149505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:26:35.149505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:26:35.149505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:26:35.149505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:26:35.149505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:26:35.149505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:35.149505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:35.152718 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 257/527] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:35.154351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7db46-3d6b-4f63-9200-f96a5fb0b8fd None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:35.158502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:35.195993 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ebd8e-94d7-43b0-a6a0-de7a352f48de None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:35.233142 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163050a-25e3-4dad-9ddd-62a5e4dd515c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:35.234144 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163050a-25e3-4dad-9ddd-62a5e4dd515c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:35.234647 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163050a-25e3-4dad-9ddd-62a5e4dd515c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:35.235133 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163050a-25e3-4dad-9ddd-62a5e4dd515c None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:35.236122 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5163050a-25e3-4dad-9ddd-62a5e4dd515c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:35.236621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5163050a-25e3-4dad-9ddd-62a5e4dd515c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:35.239073 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5163050a-25e3-4dad-9ddd-62a5e4dd515c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:35.259449 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 271/528] 10.222.0.47 () {56 vars in 1139 bytes} [Thu Aug 4 04:26:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:35.260595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163050a-25e3-4dad-9ddd-62a5e4dd515c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:35.283568 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7db46-3d6b-4f63-9200-f96a5fb0b8fd None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:35.321429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070da53-4d61-40d7-9aae-a892edea5b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:35.322099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070da53-4d61-40d7-9aae-a892edea5b3e None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:35.322379 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070da53-4d61-40d7-9aae-a892edea5b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:35.322651 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070da53-4d61-40d7-9aae-a892edea5b3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:35.333759 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070da53-4d61-40d7-9aae-a892edea5b3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:35.335349 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070da53-4d61-40d7-9aae-a892edea5b3e None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:35.351945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-4070da53-4d61-40d7-9aae-a892edea5b3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:35.351945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:35.351945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:35.351945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:35.351945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:35.351945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:35.351945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:35.351945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:35.351945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:35.351945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:35.351945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:35.351945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:35.351945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:35.351945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:35.351945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:35.352966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:35.352966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:35.352966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:35.352966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:35.352966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:35.352966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:35.352966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:35.352966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:35.352966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:35.352966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:35.352966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:35.352966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:35.352966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:35.352966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:35.352966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:35.352966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:35.352966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:35.353830 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:35.353830 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:35.353830 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:35.353830 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:35.353830 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 258/529] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:35.354912 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070da53-4d61-40d7-9aae-a892edea5b3e None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:35.379846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163050a-25e3-4dad-9ddd-62a5e4dd515c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:35.392077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:35.417197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:35.420477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ccaf1-fc49-4ca6-b51a-f4ac435ed4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:35.421620 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ccaf1-fc49-4ca6-b51a-f4ac435ed4fa None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:35.422083 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ccaf1-fc49-4ca6-b51a-f4ac435ed4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:35.422506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ccaf1-fc49-4ca6-b51a-f4ac435ed4fa None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:35.423403 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ccaf1-fc49-4ca6-b51a-f4ac435ed4fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:35.423862 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ccaf1-fc49-4ca6-b51a-f4ac435ed4fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:35.426428 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ccaf1-fc49-4ca6-b51a-f4ac435ed4fa None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:35.447011 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 272/530] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:26:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:35.448508 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ccaf1-fc49-4ca6-b51a-f4ac435ed4fa None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:35.464563 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070da53-4d61-40d7-9aae-a892edea5b3e None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:35.504673 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f6d8e-7e55-490e-a6f0-ebdf0010f0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:35.505297 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f6d8e-7e55-490e-a6f0-ebdf0010f0be None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:35.505625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f6d8e-7e55-490e-a6f0-ebdf0010f0be None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:35.505939 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f6d8e-7e55-490e-a6f0-ebdf0010f0be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:35.582484 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8f6d8e-7e55-490e-a6f0-ebdf0010f0be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:35.584375 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8f6d8e-7e55-490e-a6f0-ebdf0010f0be None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:35.588076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ccaf1-fc49-4ca6-b51a-f4ac435ed4fa None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:35.628334 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4462830-3e0f-4947-a68f-946be94b4a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:35.628961 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4462830-3e0f-4947-a68f-946be94b4a97 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:35.629245 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4462830-3e0f-4947-a68f-946be94b4a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:35.629521 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4462830-3e0f-4947-a68f-946be94b4a97 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:35.630303 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4462830-3e0f-4947-a68f-946be94b4a97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:35.630558 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4462830-3e0f-4947-a68f-946be94b4a97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:35.632340 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4462830-3e0f-4947-a68f-946be94b4a97 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:35.645154 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 273/531] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:35.646101 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4462830-3e0f-4947-a68f-946be94b4a97 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:35.651025 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:35.680487 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:35.704270 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 259/532] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:26:35.705353 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f6d8e-7e55-490e-a6f0-ebdf0010f0be None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:35.759305 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4462830-3e0f-4947-a68f-946be94b4a97 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:35.797289 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf52fc-333f-409f-b0a7-dfe972fffe75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:35.798374 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf52fc-333f-409f-b0a7-dfe972fffe75 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:35.798836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf52fc-333f-409f-b0a7-dfe972fffe75 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:35.799288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf52fc-333f-409f-b0a7-dfe972fffe75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:35.800288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf52fc-333f-409f-b0a7-dfe972fffe75 None admin] RBAC: Adding query filter params (role.id=cc86b64693b6417e90ad9361460574b0, scope.project.id=f37a5f3140ec431da8ea14568654114a, user.id=b77a955866c54a43a7aaa271fefdec57) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:35.800937 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf52fc-333f-409f-b0a7-dfe972fffe75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:35.802612 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf52fc-333f-409f-b0a7-dfe972fffe75 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:35.817577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f6d8e-7e55-490e-a6f0-ebdf0010f0be None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:35.818001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 274/533] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:26:35 2022] GET /identity/v3/role_assignments?user.id=b77a955866c54a43a7aaa271fefdec57&scope.project.id=f37a5f3140ec431da8ea14568654114a&role.id=cc86b64693b6417e90ad9361460574b0 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:35.819346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf52fc-333f-409f-b0a7-dfe972fffe75 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:35.853496 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcbff4-0855-46f2-ab2c-0c80a33bd5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:35.854521 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcbff4-0855-46f2-ab2c-0c80a33bd5e1 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:35.854680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcbff4-0855-46f2-ab2c-0c80a33bd5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:35.855072 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcbff4-0855-46f2-ab2c-0c80a33bd5e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:35.863571 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dcbff4-0855-46f2-ab2c-0c80a33bd5e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:35.873577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dcbff4-0855-46f2-ab2c-0c80a33bd5e1 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:35.880076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-f5dcbff4-0855-46f2-ab2c-0c80a33bd5e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:35.880076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:35.880076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:35.880076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:35.880076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:35.880076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:35.880076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:35.880076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:35.880076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:35.880076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:35.880076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:35.880076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:35.880076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:26:35.880076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:26:35.880076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:26:35.881126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:26:35.881126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:35.881126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:35.881126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:35.881126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:35.881126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:35.881126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:35.881126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:35.881126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:35.881126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:35.881126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:35.881126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:35.881126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:35.881126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:35.881126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:35.881126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:35.881126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:35.882222 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:26:35.882222 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:26:35.882222 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:26:35.882222 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:26:35.882222 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:26:35.882222 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:26:35.882222 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:35.882222 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:35.884565 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 260/534] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:35.890608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcbff4-0855-46f2-ab2c-0c80a33bd5e1 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:35.927800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:35.955988 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:35.966408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf52fc-333f-409f-b0a7-dfe972fffe75 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:36.014714 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83a3d0-5415-4ef9-9d3f-6bb1ca193350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:36.016168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83a3d0-5415-4ef9-9d3f-6bb1ca193350 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:36.016690 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83a3d0-5415-4ef9-9d3f-6bb1ca193350 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:36.017143 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83a3d0-5415-4ef9-9d3f-6bb1ca193350 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:36.030140 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a83a3d0-5415-4ef9-9d3f-6bb1ca193350 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:36.031993 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a83a3d0-5415-4ef9-9d3f-6bb1ca193350 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:36.050238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-9a83a3d0-5415-4ef9-9d3f-6bb1ca193350 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:36.050238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:36.050238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:36.050238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:36.050238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:36.050238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:36.050238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:36.050238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:36.050238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:36.050238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:36.050238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:36.050238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:36.050238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:36.050238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:36.050238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:36.051275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:36.051275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:36.051275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:36.051275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:36.051275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:36.051275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:36.051275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:36.051275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:36.051275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:36.051275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:36.051275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:36.051275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:36.051275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:36.051275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:36.051275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:36.051275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:36.051275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:36.052094 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:36.052094 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:36.052094 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:36.052094 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:36.054459 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 275/535] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:36.068177 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83a3d0-5415-4ef9-9d3f-6bb1ca193350 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:36.120698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcbff4-0855-46f2-ab2c-0c80a33bd5e1 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:36.201177 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83a3d0-5415-4ef9-9d3f-6bb1ca193350 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:36.234849 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723e60c-b72c-4890-bc13-3b49a1db357e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:36.235483 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723e60c-b72c-4890-bc13-3b49a1db357e None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:36.235764 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723e60c-b72c-4890-bc13-3b49a1db357e None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:36.236035 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723e60c-b72c-4890-bc13-3b49a1db357e None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:36.237321 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723e60c-b72c-4890-bc13-3b49a1db357e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:36.237657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723e60c-b72c-4890-bc13-3b49a1db357e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:36.239095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723e60c-b72c-4890-bc13-3b49a1db357e None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:36.243622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea88c9e-2700-40be-b9c5-7188e74e7649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:36.244470 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea88c9e-2700-40be-b9c5-7188e74e7649 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:36.244951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea88c9e-2700-40be-b9c5-7188e74e7649 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:36.245430 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea88c9e-2700-40be-b9c5-7188e74e7649 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:36.246342 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea88c9e-2700-40be-b9c5-7188e74e7649 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:36.246852 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea88c9e-2700-40be-b9c5-7188e74e7649 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:36.248401 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea88c9e-2700-40be-b9c5-7188e74e7649 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:36.262789 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 276/536] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:36.276404 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea88c9e-2700-40be-b9c5-7188e74e7649 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:36.278389 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 261/537] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:26:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:36.283126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723e60c-b72c-4890-bc13-3b49a1db357e None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:36.288894 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:36.364828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:36.399366 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723e60c-b72c-4890-bc13-3b49a1db357e None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:36.438940 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62aacda-61dd-4fd2-8aad-f7628598d80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:36.439552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62aacda-61dd-4fd2-8aad-f7628598d80c None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:36.439831 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62aacda-61dd-4fd2-8aad-f7628598d80c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:36.440181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62aacda-61dd-4fd2-8aad-f7628598d80c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:36.441166 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62aacda-61dd-4fd2-8aad-f7628598d80c None admin] RBAC: Adding query filter params (role.id=cc86b64693b6417e90ad9361460574b0, scope.project.id=f37a5f3140ec431da8ea14568654114a, user.id=8b1d67b28dca49ca9cb0ff28cad62aa1) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:36.441474 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62aacda-61dd-4fd2-8aad-f7628598d80c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:36.442166 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea88c9e-2700-40be-b9c5-7188e74e7649 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:36.442923 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62aacda-61dd-4fd2-8aad-f7628598d80c None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:36.449157 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 262/538] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:26:36 2022] GET /identity/v3/role_assignments?user.id=8b1d67b28dca49ca9cb0ff28cad62aa1&scope.project.id=f37a5f3140ec431da8ea14568654114a&role.id=cc86b64693b6417e90ad9361460574b0 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:36.450359 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6601d3a-e5b8-4b13-af7f-35cfced675f6 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:36.450646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6601d3a-e5b8-4b13-af7f-35cfced675f6 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:36.450922 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6601d3a-e5b8-4b13-af7f-35cfced675f6 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:36.458567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 263/539] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:36 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:36.472084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51c949-eb8c-415f-9cee-cfb84ce162f1 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:36.472526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51c949-eb8c-415f-9cee-cfb84ce162f1 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:36.472798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51c949-eb8c-415f-9cee-cfb84ce162f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:36.486470 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1318dc9-37c1-47d7-a957-a379b3739078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:36.487251 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1318dc9-37c1-47d7-a957-a379b3739078 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:36.487540 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1318dc9-37c1-47d7-a957-a379b3739078 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:36.487821 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1318dc9-37c1-47d7-a957-a379b3739078 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:36.496399 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1318dc9-37c1-47d7-a957-a379b3739078 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:36.497908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1318dc9-37c1-47d7-a957-a379b3739078 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:36.513400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-e1318dc9-37c1-47d7-a957-a379b3739078 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:36.513400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:36.513400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:36.513400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:36.513400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:36.513400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:36.513400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:36.513400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:36.513400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:36.513400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:36.513400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:36.513400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:36.513400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:26:36.513400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:26:36.513400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:26:36.514375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:26:36.514375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:36.514375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:36.514375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:36.514375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:36.514375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:36.514375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:36.514375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:36.514375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:36.514375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:36.514375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:36.514375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:36.514375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:36.514375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:36.514375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:36.514375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:36.514375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:36.515187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:26:36.515187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:26:36.515187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:26:36.515187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:26:36.515187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:26:36.515187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:26:36.515187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:36.515187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:36.515187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 277/540] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:36.518162 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1318dc9-37c1-47d7-a957-a379b3739078 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:36.533252 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-df51c949-eb8c-415f-9cee-cfb84ce162f1 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:36.534563 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-df51c949-eb8c-415f-9cee-cfb84ce162f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:36.643203 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:36.681646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 264/541] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:36.701875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1318dc9-37c1-47d7-a957-a379b3739078 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:36.705309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51c949-eb8c-415f-9cee-cfb84ce162f1 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:36.744881 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e32a63-79bb-4acb-b05a-31d95e98b260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:36.745937 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e32a63-79bb-4acb-b05a-31d95e98b260 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:36.746443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e32a63-79bb-4acb-b05a-31d95e98b260 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:36.746954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e32a63-79bb-4acb-b05a-31d95e98b260 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:36.748008 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e32a63-79bb-4acb-b05a-31d95e98b260 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:36.748517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e32a63-79bb-4acb-b05a-31d95e98b260 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:36.750198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e32a63-79bb-4acb-b05a-31d95e98b260 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:36.764964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 278/542] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:26:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:36.770975 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e32a63-79bb-4acb-b05a-31d95e98b260 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:36.817797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:36.886860 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51c949-eb8c-415f-9cee-cfb84ce162f1 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:36.943446 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e32a63-79bb-4acb-b05a-31d95e98b260 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:36.973906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:36.982193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361008d7-209f-4171-82ee-f9a805528540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:36.982836 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361008d7-209f-4171-82ee-f9a805528540 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:36.983114 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361008d7-209f-4171-82ee-f9a805528540 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:36.983383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361008d7-209f-4171-82ee-f9a805528540 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:36.987792 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67de11-a02a-407c-93f8-42ebae2c7622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:36.988779 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67de11-a02a-407c-93f8-42ebae2c7622 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:36.989401 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67de11-a02a-407c-93f8-42ebae2c7622 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:36.989892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67de11-a02a-407c-93f8-42ebae2c7622 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:36.990880 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67de11-a02a-407c-93f8-42ebae2c7622 None admin] RBAC: Adding query filter params (role.id=cc86b64693b6417e90ad9361460574b0, scope.project.id=f37a5f3140ec431da8ea14568654114a, user.id=1a2a647f97fc4d369f562e8b5a4bf68c) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:36.991649 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67de11-a02a-407c-93f8-42ebae2c7622 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:36.996447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67de11-a02a-407c-93f8-42ebae2c7622 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:37.006666 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 279/543] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:26:36 2022] GET /identity/v3/role_assignments?user.id=1a2a647f97fc4d369f562e8b5a4bf68c&scope.project.id=f37a5f3140ec431da8ea14568654114a&role.id=cc86b64693b6417e90ad9361460574b0 => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:37.010626 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361008d7-209f-4171-82ee-f9a805528540 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:37.016343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-361008d7-209f-4171-82ee-f9a805528540 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:37.016343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:37.016343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:37.016343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:37.016343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:37.016343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:37.016343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:37.016343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:37.016343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:37.016343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:37.016343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:37.016343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:37.016343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:37.016343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:37.016343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:37.016343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:37.017290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:37.017290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:37.017290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:37.017290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:37.017290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:37.017290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:37.017290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:37.017290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:37.017290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:37.017290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:37.017290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:37.017290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:37.017290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:37.017290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:37.017290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:37.017290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:37.018635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:37.018635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:37.018635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:37.018635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:37.018635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:37.018635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:37.018635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:37.018635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:37.018635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:37.018635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:37.018635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:37.018635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:37.024757 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 265/544] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Aug 4 04:26:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:26:37.027988 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67de11-a02a-407c-93f8-42ebae2c7622 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:37.141594 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67de11-a02a-407c-93f8-42ebae2c7622 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:37.180283 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00f173-ac2b-41ca-8ce3-e94b7bf48651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:37.181328 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00f173-ac2b-41ca-8ce3-e94b7bf48651 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:37.181859 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00f173-ac2b-41ca-8ce3-e94b7bf48651 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:37.182340 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00f173-ac2b-41ca-8ce3-e94b7bf48651 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:37.183332 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00f173-ac2b-41ca-8ce3-e94b7bf48651 None admin] RBAC: Adding query filter params (name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:37.183854 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00f173-ac2b-41ca-8ce3-e94b7bf48651 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:37.185497 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00f173-ac2b-41ca-8ce3-e94b7bf48651 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:37.200212 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 280/545] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Aug 4 04:26:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:37.201854 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361008d7-209f-4171-82ee-f9a805528540 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:37.218259 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de95bf4-b9c5-4eb3-9bb7-e9babbb84614 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:37.218879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de95bf4-b9c5-4eb3-9bb7-e9babbb84614 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:37.219380 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de95bf4-b9c5-4eb3-9bb7-e9babbb84614 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:37.221334 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 281/546] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:37.229671 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48814f-f537-4a51-99f4-3f0944d18c4f None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:37.229671 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48814f-f537-4a51-99f4-3f0944d18c4f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:37.229671 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48814f-f537-4a51-99f4-3f0944d18c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:37.258948 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-bf48814f-f537-4a51-99f4-3f0944d18c4f None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:37.260892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-bf48814f-f537-4a51-99f4-3f0944d18c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:37.325951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:37.344644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 282/547] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:26:37.365536 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361008d7-209f-4171-82ee-f9a805528540 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:37.409393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48814f-f537-4a51-99f4-3f0944d18c4f None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:37.410814 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:37.420472 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8436e5d-1527-4fc7-b801-9efa049a346f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:37.421506 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8436e5d-1527-4fc7-b801-9efa049a346f None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:37.425668 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8436e5d-1527-4fc7-b801-9efa049a346f None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:37.425668 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8436e5d-1527-4fc7-b801-9efa049a346f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:37.446687 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8436e5d-1527-4fc7-b801-9efa049a346f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:37.450210 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8436e5d-1527-4fc7-b801-9efa049a346f None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:37.472219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-a8436e5d-1527-4fc7-b801-9efa049a346f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:37.472219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:37.472219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:37.472219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:37.472219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:37.472219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:37.472219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:37.472219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:37.472219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:37.472219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:37.472219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:37.472219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:37.472219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:37.472219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:37.472219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:37.473089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:37.473089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:37.473089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:37.473089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:37.473089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:37.473089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:37.473089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:37.473089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:37.473089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:37.473089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:37.473089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:37.473089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:37.473089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:37.473089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:37.473089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:37.473089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:37.473089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:37.473948 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:37.473948 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:37.473948 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:37.473948 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:37.473948 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 266/548] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:37.479453 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8436e5d-1527-4fc7-b801-9efa049a346f None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:37.548833 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48814f-f537-4a51-99f4-3f0944d18c4f None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:37.601106 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4b3a7-6645-4d51-b901-a33f03aa94ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:37.601781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4b3a7-6645-4d51-b901-a33f03aa94ff None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:37.602061 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4b3a7-6645-4d51-b901-a33f03aa94ff None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:37.602329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4b3a7-6645-4d51-b901-a33f03aa94ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:37.621053 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4b3a7-6645-4d51-b901-a33f03aa94ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:37.634736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-37f4b3a7-6645-4d51-b901-a33f03aa94ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:37.634736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:37.634736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:37.634736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:37.634736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:37.634736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:37.634736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:37.634736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:37.634736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:37.634736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:37.634736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:37.634736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:37.634736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:37.634736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:37.634736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:37.634736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:37.635740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:37.635740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:37.635740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:37.635740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:37.635740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:37.635740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:37.635740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:37.635740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:37.635740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:37.635740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:37.635740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:37.635740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:37.635740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:37.635740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:37.635740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:37.635740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:37.637003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:37.637003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:37.637003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:37.637003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:37.637003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:37.637003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:37.637003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:37.637003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:37.637003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:37.637003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:37.637003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:37.637003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:37.639965 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 283/549] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Aug 4 04:26:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:26:37.658276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4b3a7-6645-4d51-b901-a33f03aa94ff None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:37.677600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8436e5d-1527-4fc7-b801-9efa049a346f None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:37.714242 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeac4bf-c9b3-44cc-8117-9b6a2b84f649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:37.715004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeac4bf-c9b3-44cc-8117-9b6a2b84f649 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:37.715292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeac4bf-c9b3-44cc-8117-9b6a2b84f649 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:37.715553 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeac4bf-c9b3-44cc-8117-9b6a2b84f649 None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:37.716246 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeac4bf-c9b3-44cc-8117-9b6a2b84f649 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:37.716546 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeac4bf-c9b3-44cc-8117-9b6a2b84f649 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:37.718200 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeac4bf-c9b3-44cc-8117-9b6a2b84f649 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:37.732159 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 267/550] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:37.737738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:37.738994 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeac4bf-c9b3-44cc-8117-9b6a2b84f649 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:37.788712 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:37.880237 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4b3a7-6645-4d51-b901-a33f03aa94ff None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:37.907342 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeac4bf-c9b3-44cc-8117-9b6a2b84f649 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:37.954752 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859aeb5-5fc2-4f71-959f-f5526c315cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:37.955670 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859aeb5-5fc2-4f71-959f-f5526c315cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:37.956092 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859aeb5-5fc2-4f71-959f-f5526c315cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:37.956591 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859aeb5-5fc2-4f71-959f-f5526c315cd2 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:37.958134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e859aeb5-5fc2-4f71-959f-f5526c315cd2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:37.959340 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e859aeb5-5fc2-4f71-959f-f5526c315cd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:37.960889 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e859aeb5-5fc2-4f71-959f-f5526c315cd2 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:37.979239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 284/551] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Aug 4 04:26:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:37.986063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de29b03-2278-40e2-bd1a-917ce269333e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:37.987491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859aeb5-5fc2-4f71-959f-f5526c315cd2 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:37.988700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de29b03-2278-40e2-bd1a-917ce269333e None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:37.989886 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de29b03-2278-40e2-bd1a-917ce269333e None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:37.990993 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de29b03-2278-40e2-bd1a-917ce269333e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:38.017662 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de29b03-2278-40e2-bd1a-917ce269333e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:38.024727 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de29b03-2278-40e2-bd1a-917ce269333e None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:38.071445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-7de29b03-2278-40e2-bd1a-917ce269333e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 04:26:38.071445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:38.071445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:38.071445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:38.071445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:38.071445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:38.071445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:38.071445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:38.071445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:38.071445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:38.071445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:38.071445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:38.071445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:26:38.071445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:26:38.071445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:26:38.071445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:26:38.073476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:38.073476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:38.073476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:26:38.073476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:38.073476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:26:38.073476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:38.073476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:38.073476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:38.073476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:38.073476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:38.073476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:38.073476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:38.073476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:38.073476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:38.073476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:38.073476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:38.073476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:38.073476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:38.075301 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:38.075301 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:38.075301 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:26:38.075301 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:26:38.075301 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:26:38.075301 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:26:38.075301 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:26:38.075301 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:26:38.075301 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 04:26:38.075301 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:38.078604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 268/552] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:26:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:38.085468 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de29b03-2278-40e2-bd1a-917ce269333e None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:38.188271 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859aeb5-5fc2-4f71-959f-f5526c315cd2 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:38.191493 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:38.227242 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439e75c-4a27-4f14-94d5-23e423669574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:38.228138 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439e75c-4a27-4f14-94d5-23e423669574 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:38.228588 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439e75c-4a27-4f14-94d5-23e423669574 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:38.229045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439e75c-4a27-4f14-94d5-23e423669574 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:38.242166 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439e75c-4a27-4f14-94d5-23e423669574 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:38.243874 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439e75c-4a27-4f14-94d5-23e423669574 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:38.259920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-e439e75c-4a27-4f14-94d5-23e423669574 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:38.259920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:38.259920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:38.259920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:38.259920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:38.259920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:38.259920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:38.259920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:38.259920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:38.259920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:38.259920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:38.259920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:38.259920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:38.259920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:38.259920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:38.261007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:38.261007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:38.261007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:38.261007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:38.261007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:38.261007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:38.261007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:38.261007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:38.261007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:38.261007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:38.261007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:38.261007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:38.261007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:38.261007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:38.261007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:38.261007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:38.261007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:38.262171 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:38.262171 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:38.262171 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:38.262171 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:38.263949 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 285/553] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:38.269791 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439e75c-4a27-4f14-94d5-23e423669574 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:38.347219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:38.351036 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de29b03-2278-40e2-bd1a-917ce269333e None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:38.391862 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439e75c-4a27-4f14-94d5-23e423669574 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:38.392771 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f092b4-607e-4ded-89a3-86ee98ecd272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:38.393873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f092b4-607e-4ded-89a3-86ee98ecd272 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:38.395233 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f092b4-607e-4ded-89a3-86ee98ecd272 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:38.395671 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f092b4-607e-4ded-89a3-86ee98ecd272 None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:38.396647 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f092b4-607e-4ded-89a3-86ee98ecd272 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:38.397095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f092b4-607e-4ded-89a3-86ee98ecd272 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:38.398716 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f092b4-607e-4ded-89a3-86ee98ecd272 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:38.421629 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 269/554] 10.222.0.47 () {56 vars in 1135 bytes} [Thu Aug 4 04:26:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:38.428267 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa0a47-7e4a-4942-9806-6bd5e59f313b None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:38.428559 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa0a47-7e4a-4942-9806-6bd5e59f313b None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:38.429098 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa0a47-7e4a-4942-9806-6bd5e59f313b None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:38.430822 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 270/555] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:38.431699 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa0a47-7e4a-4942-9806-6bd5e59f313b None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:38.434787 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d654d2-3d41-4291-bb9e-33f8550622a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:38.435648 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d654d2-3d41-4291-bb9e-33f8550622a7 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:38.436967 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d654d2-3d41-4291-bb9e-33f8550622a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:38.437424 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d654d2-3d41-4291-bb9e-33f8550622a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:38.438395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d654d2-3d41-4291-bb9e-33f8550622a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:38.438934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d654d2-3d41-4291-bb9e-33f8550622a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:38.440621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d654d2-3d41-4291-bb9e-33f8550622a7 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:38.459222 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 286/556] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:38.461368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33a6a2-0c0f-4fd0-9ba9-2a8e345374cc None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:38.462156 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33a6a2-0c0f-4fd0-9ba9-2a8e345374cc None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:38.462609 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33a6a2-0c0f-4fd0-9ba9-2a8e345374cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:38.486985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-2f33a6a2-0c0f-4fd0-9ba9-2a8e345374cc None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:38.488663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2f33a6a2-0c0f-4fd0-9ba9-2a8e345374cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:38.507320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:38.567216 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa0a47-7e4a-4942-9806-6bd5e59f313b None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:38.583842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 287/557] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:38.584966 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33a6a2-0c0f-4fd0-9ba9-2a8e345374cc None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:38.618963 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6de98f-08fc-41bf-9740-5c32f69b5e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:38.619878 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6de98f-08fc-41bf-9740-5c32f69b5e9b None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:38.620554 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6de98f-08fc-41bf-9740-5c32f69b5e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:38.621466 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6de98f-08fc-41bf-9740-5c32f69b5e9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:38.660028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:38.700945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33a6a2-0c0f-4fd0-9ba9-2a8e345374cc None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:38.714290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6de98f-08fc-41bf-9740-5c32f69b5e9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:38.716031 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6de98f-08fc-41bf-9740-5c32f69b5e9b None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:38.738627 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea14a1c-bc23-47e0-b21d-bfcc78f90785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:38.739511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea14a1c-bc23-47e0-b21d-bfcc78f90785 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:38.739951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea14a1c-bc23-47e0-b21d-bfcc78f90785 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:38.740328 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea14a1c-bc23-47e0-b21d-bfcc78f90785 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:38.748394 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea14a1c-bc23-47e0-b21d-bfcc78f90785 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:38.750147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea14a1c-bc23-47e0-b21d-bfcc78f90785 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:38.763842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-bea14a1c-bc23-47e0-b21d-bfcc78f90785 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 04:26:38.763842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:38.763842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:38.763842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:38.763842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:38.763842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:38.763842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:38.763842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:38.763842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:38.763842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:38.763842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:38.763842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:38.763842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:26:38.763842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:26:38.763842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:26:38.763842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:26:38.765286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:38.765286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:38.765286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:26:38.765286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:38.765286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:26:38.765286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:38.765286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:38.765286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:38.765286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:38.765286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:38.765286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:38.765286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:38.765286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:38.765286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:38.765286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:38.765286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:38.765286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:38.765286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:38.766228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:38.766228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:38.766228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:26:38.766228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:26:38.766228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:26:38.766228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:26:38.766228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:26:38.766228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:26:38.766228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 04:26:38.766228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:38.766228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 288/558] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Aug 4 04:26:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:38.766825 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea14a1c-bc23-47e0-b21d-bfcc78f90785 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:38.777233 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:38.831420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 271/559] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 04:26:38.832629 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6de98f-08fc-41bf-9740-5c32f69b5e9b None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:38.882906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea14a1c-bc23-47e0-b21d-bfcc78f90785 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:38.895364 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:38.919087 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f41796-8066-4d3a-851d-d5a702a98745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:38.919724 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f41796-8066-4d3a-851d-d5a702a98745 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:38.920007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f41796-8066-4d3a-851d-d5a702a98745 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:38.920282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f41796-8066-4d3a-851d-d5a702a98745 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:38.933063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f41796-8066-4d3a-851d-d5a702a98745 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:38.938506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-71f41796-8066-4d3a-851d-d5a702a98745 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:38.938506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:38.938506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:38.938506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:38.938506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:38.938506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:38.938506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:38.938506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:38.938506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:38.938506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:38.938506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:38.938506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:38.938506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:38.938506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:38.938506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:38.938506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:38.939479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:38.939479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:38.939479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:38.939479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:38.939479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:38.939479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:38.939479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:38.939479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:38.939479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:38.939479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:38.939479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:38.939479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:38.939479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:38.939479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:38.939479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:38.939479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:38.941097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:38.941097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:38.941097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:38.941097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:38.941097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:38.941097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:38.941097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:38.941097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:38.941097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:38.941097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:38.941097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:38.941097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:38.941097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 289/560] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Aug 4 04:26:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:38.941097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f41796-8066-4d3a-851d-d5a702a98745 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:38.949515 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6de98f-08fc-41bf-9740-5c32f69b5e9b None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:38.997001 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90cce9-e034-43a4-af59-eefc2549fe68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:38.997627 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90cce9-e034-43a4-af59-eefc2549fe68 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:38.997832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90cce9-e034-43a4-af59-eefc2549fe68 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:38.998087 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90cce9-e034-43a4-af59-eefc2549fe68 None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:38.998881 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90cce9-e034-43a4-af59-eefc2549fe68 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:38.999187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90cce9-e034-43a4-af59-eefc2549fe68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:39.000585 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90cce9-e034-43a4-af59-eefc2549fe68 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:39.015937 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:39.027741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 272/561] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Aug 4 04:26:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:39.027741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d478434-1298-4509-8db4-1527ff449d92 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:39.027741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d478434-1298-4509-8db4-1527ff449d92 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:39.027741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d478434-1298-4509-8db4-1527ff449d92 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:39.027741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 273/562] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:39.027741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d478434-1298-4509-8db4-1527ff449d92 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:39.059979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f41796-8066-4d3a-851d-d5a702a98745 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:39.100259 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86444f0-0477-4360-abe9-f71d6ff7351c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:39.100946 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86444f0-0477-4360-abe9-f71d6ff7351c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:39.101230 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86444f0-0477-4360-abe9-f71d6ff7351c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:39.101635 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86444f0-0477-4360-abe9-f71d6ff7351c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:39.112333 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86444f0-0477-4360-abe9-f71d6ff7351c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:39.113839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86444f0-0477-4360-abe9-f71d6ff7351c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:39.128602 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-b86444f0-0477-4360-abe9-f71d6ff7351c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:39.128602 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:39.128602 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:39.128602 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:39.128602 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:39.128602 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:39.128602 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:39.128602 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:39.128602 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:39.128602 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:39.128602 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:39.128602 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:39.128602 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:39.128602 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:39.128602 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:39.129584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:39.129584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:39.129584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:39.129584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:39.129584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:39.129584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:39.129584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:39.129584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:39.129584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:39.129584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:39.129584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:39.129584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:39.129584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:39.129584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:39.129584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:39.129584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:39.129584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:39.130483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:39.130483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:39.130483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:39.130483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:39.130483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 290/563] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:39.130752 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86444f0-0477-4360-abe9-f71d6ff7351c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:39.146875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d478434-1298-4509-8db4-1527ff449d92 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:39.167404 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:39.189085 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f1ece-b1f4-4332-b827-b1d516e4c7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:39.189754 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f1ece-b1f4-4332-b827-b1d516e4c7f3 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:39.190655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f1ece-b1f4-4332-b827-b1d516e4c7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:39.190840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f1ece-b1f4-4332-b827-b1d516e4c7f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:39.191549 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20f1ece-b1f4-4332-b827-b1d516e4c7f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:39.191847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20f1ece-b1f4-4332-b827-b1d516e4c7f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:39.193198 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20f1ece-b1f4-4332-b827-b1d516e4c7f3 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:39.201869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 274/564] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Aug 4 04:26:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:39.203125 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abccc7b-2604-47d7-a542-6bf3f34c7f02 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:39.203433 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abccc7b-2604-47d7-a542-6bf3f34c7f02 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:39.203701 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abccc7b-2604-47d7-a542-6bf3f34c7f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:39.221279 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-0abccc7b-2604-47d7-a542-6bf3f34c7f02 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:39.222629 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-0abccc7b-2604-47d7-a542-6bf3f34c7f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:39.264289 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86444f0-0477-4360-abe9-f71d6ff7351c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:39.282250 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:39.296099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 275/565] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:39.296895 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abccc7b-2604-47d7-a542-6bf3f34c7f02 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:39.312413 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72edd745-10f9-4726-86a1-ff193143c545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:39.313940 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72edd745-10f9-4726-86a1-ff193143c545 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:39.314673 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72edd745-10f9-4726-86a1-ff193143c545 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:39.316559 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72edd745-10f9-4726-86a1-ff193143c545 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:39.332136 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72edd745-10f9-4726-86a1-ff193143c545 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:39.333799 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72edd745-10f9-4726-86a1-ff193143c545 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:39.351168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-72edd745-10f9-4726-86a1-ff193143c545 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:39.351168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:39.351168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:39.351168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:39.351168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:39.351168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:39.351168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:39.351168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:39.351168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:39.351168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:39.351168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:39.351168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:39.351168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:39.351168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:39.351168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:39.352072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:39.352072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:39.352072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:39.352072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:39.352072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:39.352072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:39.352072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:39.352072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:39.352072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:39.352072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:39.352072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:39.352072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:39.352072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:39.352072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:39.352072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:39.352072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:39.352072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:39.352951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:39.352951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:39.352951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:39.352951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:39.352951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 291/566] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:39.353230 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72edd745-10f9-4726-86a1-ff193143c545 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:39.407068 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abccc7b-2604-47d7-a542-6bf3f34c7f02 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:39.418112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:39.444732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc89e2-6a85-4689-8de7-ed7fa943e040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:39.445352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc89e2-6a85-4689-8de7-ed7fa943e040 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:39.445670 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc89e2-6a85-4689-8de7-ed7fa943e040 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:39.445995 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc89e2-6a85-4689-8de7-ed7fa943e040 None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:39.446704 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc89e2-6a85-4689-8de7-ed7fa943e040 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:39.447073 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc89e2-6a85-4689-8de7-ed7fa943e040 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:39.448447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc89e2-6a85-4689-8de7-ed7fa943e040 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:39.461442 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 276/567] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:39.462401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc89e2-6a85-4689-8de7-ed7fa943e040 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:39.467716 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72edd745-10f9-4726-86a1-ff193143c545 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:39.507399 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e125aa2-a7b6-4e89-b888-e7a1d3aaa04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:39.508203 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e125aa2-a7b6-4e89-b888-e7a1d3aaa04e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:39.508486 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e125aa2-a7b6-4e89-b888-e7a1d3aaa04e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:39.508760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e125aa2-a7b6-4e89-b888-e7a1d3aaa04e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:39.520686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e125aa2-a7b6-4e89-b888-e7a1d3aaa04e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:39.522181 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e125aa2-a7b6-4e89-b888-e7a1d3aaa04e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:39.536350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-6e125aa2-a7b6-4e89-b888-e7a1d3aaa04e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:39.536350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:39.536350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:39.536350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:39.536350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:39.536350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:39.536350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:39.536350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:39.536350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:39.536350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:39.536350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:39.536350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:39.536350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:39.536350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:39.536350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:39.537338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:39.537338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:39.537338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:39.537338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:39.537338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:39.537338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:39.537338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:39.537338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:39.537338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:39.537338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:39.537338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:39.537338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:39.537338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:39.537338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:39.537338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:39.537338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:39.537338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:39.538240 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:39.538240 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:39.538240 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:39.538240 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:39.538240 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 292/568] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:39.538520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e125aa2-a7b6-4e89-b888-e7a1d3aaa04e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:39.563072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:39.573742 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc89e2-6a85-4689-8de7-ed7fa943e040 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:39.607187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2edc3-7d54-48e8-8d69-377808b7a2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:39.607817 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2edc3-7d54-48e8-8d69-377808b7a2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:39.608096 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2edc3-7d54-48e8-8d69-377808b7a2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:39.608367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2edc3-7d54-48e8-8d69-377808b7a2c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:39.618504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f2edc3-7d54-48e8-8d69-377808b7a2c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:39.623851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-71f2edc3-7d54-48e8-8d69-377808b7a2c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:39.623851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:39.623851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:39.623851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:39.623851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:39.623851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:39.623851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:39.623851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:39.623851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:39.623851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:39.623851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:39.623851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:39.623851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:39.623851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:39.623851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:39.623851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:39.624741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:39.624741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:39.624741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:39.624741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:39.624741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:39.624741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:39.624741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:39.624741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:39.624741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:39.624741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:39.624741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:39.624741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:39.624741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:39.624741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:39.624741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:39.624741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:39.625591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:39.625591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:39.625591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:39.625591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:39.625591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:39.625591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:39.625591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:39.625591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:39.625591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:39.625591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:39.625591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:39.625591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:39.625591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 277/569] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Aug 4 04:26:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:39.626471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2edc3-7d54-48e8-8d69-377808b7a2c2 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:39.647460 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e125aa2-a7b6-4e89-b888-e7a1d3aaa04e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:39.653992 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:39.682091 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8f706-565d-4baf-81f4-89d82f3bbbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:39.682702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8f706-565d-4baf-81f4-89d82f3bbbc9 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:39.682987 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8f706-565d-4baf-81f4-89d82f3bbbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:39.683262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8f706-565d-4baf-81f4-89d82f3bbbc9 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:39.683954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b8f706-565d-4baf-81f4-89d82f3bbbc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:39.684259 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b8f706-565d-4baf-81f4-89d82f3bbbc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:39.685656 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b8f706-565d-4baf-81f4-89d82f3bbbc9 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:39.697583 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 293/570] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:39.698829 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2450a-184b-42c8-9bb7-63f53fc949fc None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:39.699123 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2450a-184b-42c8-9bb7-63f53fc949fc None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:39.699397 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2450a-184b-42c8-9bb7-63f53fc949fc None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:39.700595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 294/571] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:39.701275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2450a-184b-42c8-9bb7-63f53fc949fc None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:39.728254 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2edc3-7d54-48e8-8d69-377808b7a2c2 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:39.761619 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca090c7-e9cb-47ab-8d35-bebd84d85603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:39.762244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca090c7-e9cb-47ab-8d35-bebd84d85603 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:39.762517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca090c7-e9cb-47ab-8d35-bebd84d85603 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:39.762790 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca090c7-e9cb-47ab-8d35-bebd84d85603 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:39.769601 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca090c7-e9cb-47ab-8d35-bebd84d85603 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:39.771447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca090c7-e9cb-47ab-8d35-bebd84d85603 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:39.782986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-9ca090c7-e9cb-47ab-8d35-bebd84d85603 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:39.782986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:39.782986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:39.782986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:39.782986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:39.782986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:39.782986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:39.782986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:39.782986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:39.782986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:39.782986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:39.782986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:39.782986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:26:39.782986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:26:39.782986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:26:39.784567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:26:39.784567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:39.784567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:39.784567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:39.784567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:39.784567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:39.784567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:39.784567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:39.784567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:39.784567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:39.784567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:39.784567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:39.784567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:39.784567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:39.784567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:39.784567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:39.784567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:39.785420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:26:39.785420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:26:39.785420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:26:39.785420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:26:39.785420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:26:39.785420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:26:39.785420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:39.785420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:39.785420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 278/572] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:39.785951 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca090c7-e9cb-47ab-8d35-bebd84d85603 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:39.786702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:39.809430 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2450a-184b-42c8-9bb7-63f53fc949fc None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:39.842463 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5315a94a-0283-4d6e-9e99-7f170787647c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:39.843077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315a94a-0283-4d6e-9e99-7f170787647c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:39.843358 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315a94a-0283-4d6e-9e99-7f170787647c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:39.843642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315a94a-0283-4d6e-9e99-7f170787647c None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:39.844328 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5315a94a-0283-4d6e-9e99-7f170787647c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:39.844632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5315a94a-0283-4d6e-9e99-7f170787647c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:39.846695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5315a94a-0283-4d6e-9e99-7f170787647c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:39.857637 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 295/573] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:39.858425 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5315a94a-0283-4d6e-9e99-7f170787647c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:39.875546 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:39.887680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca090c7-e9cb-47ab-8d35-bebd84d85603 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:39.921399 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72516a-49d3-4639-9ebb-9a7fd983c834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:39.922067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72516a-49d3-4639-9ebb-9a7fd983c834 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:39.922342 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72516a-49d3-4639-9ebb-9a7fd983c834 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:39.922612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72516a-49d3-4639-9ebb-9a7fd983c834 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:39.923378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb72516a-49d3-4639-9ebb-9a7fd983c834 None admin] RBAC: Adding query filter params (name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:39.923681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb72516a-49d3-4639-9ebb-9a7fd983c834 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:39.925696 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb72516a-49d3-4639-9ebb-9a7fd983c834 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:39.933802 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 279/574] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Aug 4 04:26:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:39.934757 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f4a78-2a9e-4a42-a2d5-8c7aac6ed917 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:39.935038 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f4a78-2a9e-4a42-a2d5-8c7aac6ed917 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:39.935307 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f4a78-2a9e-4a42-a2d5-8c7aac6ed917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:39.952987 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-e89f4a78-2a9e-4a42-a2d5-8c7aac6ed917 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:39.954215 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-e89f4a78-2a9e-4a42-a2d5-8c7aac6ed917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:39.960762 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5315a94a-0283-4d6e-9e99-7f170787647c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:39.994458 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be291c9c-d540-4658-8044-069a4fd4960d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:39.995084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be291c9c-d540-4658-8044-069a4fd4960d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:39.995362 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be291c9c-d540-4658-8044-069a4fd4960d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:39.995634 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be291c9c-d540-4658-8044-069a4fd4960d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:40.002031 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be291c9c-d540-4658-8044-069a4fd4960d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:40.003757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be291c9c-d540-4658-8044-069a4fd4960d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:40.014692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-be291c9c-d540-4658-8044-069a4fd4960d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:40.014692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:40.014692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:40.014692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:40.014692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:40.014692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:40.014692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:40.014692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:40.014692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:40.014692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:40.014692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:40.014692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:40.014692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:26:40.014692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:26:40.014692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:26:40.015658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:26:40.015658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:40.015658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:40.015658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:40.015658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:40.015658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:40.015658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:40.015658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:40.015658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:40.015658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:40.015658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:40.015658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:40.015658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:40.015658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:40.015658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:40.015658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:40.015658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:40.016527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:26:40.016527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:26:40.016527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:26:40.016527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:26:40.016527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:26:40.016527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:26:40.016527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:40.016527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:40.016527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 296/575] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:40.016527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be291c9c-d540-4658-8044-069a4fd4960d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:40.020082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:40.026027 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 280/576] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:40.026812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f4a78-2a9e-4a42-a2d5-8c7aac6ed917 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:40.109094 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:40.122376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be291c9c-d540-4658-8044-069a4fd4960d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:40.128915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f4a78-2a9e-4a42-a2d5-8c7aac6ed917 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:40.154733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd32b7-f652-43d7-8255-a7900ece841b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:40.155345 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd32b7-f652-43d7-8255-a7900ece841b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:40.155626 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd32b7-f652-43d7-8255-a7900ece841b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:40.155905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd32b7-f652-43d7-8255-a7900ece841b None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:40.156653 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd32b7-f652-43d7-8255-a7900ece841b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:40.156961 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd32b7-f652-43d7-8255-a7900ece841b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:40.158437 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd32b7-f652-43d7-8255-a7900ece841b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:40.163281 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa43f3-5f70-49d3-b84c-617819c55bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:40.163912 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa43f3-5f70-49d3-b84c-617819c55bee None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:40.164195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa43f3-5f70-49d3-b84c-617819c55bee None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:40.164472 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa43f3-5f70-49d3-b84c-617819c55bee None admin] PATH_INFO: `/v3/users/glance` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:40.170942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 297/577] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:26:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:40.171731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd32b7-f652-43d7-8255-a7900ece841b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:40.171858 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa43f3-5f70-49d3-b84c-617819c55bee None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:40.173281 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa43f3-5f70-49d3-b84c-617819c55bee None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:40.186681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-62aa43f3-5f70-49d3-b84c-617819c55bee None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 04:26:40.186681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:40.186681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:40.186681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:40.186681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:40.186681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:40.186681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:40.186681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:40.186681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:40.186681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:40.186681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:40.186681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:40.186681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:26:40.186681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:26:40.186681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:26:40.186681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:26:40.187665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:40.187665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:40.187665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:26:40.187665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:40.187665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:26:40.187665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:40.187665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:40.187665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:40.187665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:40.187665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:40.187665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:40.187665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:40.187665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:40.187665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:40.187665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:40.187665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:40.187665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:40.187665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:40.188600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:40.188600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:40.188600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:26:40.188600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:26:40.188600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:26:40.188600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:26:40.188600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:26:40.188600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:26:40.188600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 04:26:40.188600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:40.188600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 281/578] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:40.188600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa43f3-5f70-49d3-b84c-617819c55bee None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:40.237046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:40.277070 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd32b7-f652-43d7-8255-a7900ece841b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:40.288034 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa43f3-5f70-49d3-b84c-617819c55bee None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:40.310277 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353d51f-9a64-436d-8e1e-f62dea274b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:40.310934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353d51f-9a64-436d-8e1e-f62dea274b12 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:40.311215 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353d51f-9a64-436d-8e1e-f62dea274b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:40.311610 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353d51f-9a64-436d-8e1e-f62dea274b12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:40.321839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d353d51f-9a64-436d-8e1e-f62dea274b12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:40.322369 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8f624-f8a1-4a47-831d-8d7a85d21c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:40.322975 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8f624-f8a1-4a47-831d-8d7a85d21c8e None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:40.323255 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8f624-f8a1-4a47-831d-8d7a85d21c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:40.323343 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d353d51f-9a64-436d-8e1e-f62dea274b12 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:40.323533 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8f624-f8a1-4a47-831d-8d7a85d21c8e None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:40.324276 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8f624-f8a1-4a47-831d-8d7a85d21c8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:40.324581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8f624-f8a1-4a47-831d-8d7a85d21c8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:40.326014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8f624-f8a1-4a47-831d-8d7a85d21c8e None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:40.335636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:40.340275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-d353d51f-9a64-436d-8e1e-f62dea274b12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:40.340275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:40.340275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:40.340275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:40.340275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:40.340275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:40.340275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:40.340275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:40.340275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:40.340275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:40.340275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:40.340275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:40.340275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:40.340275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:40.340275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:40.341298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:40.341298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:40.341298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:40.341298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:40.341298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:40.341298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:40.341298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:40.341298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:40.341298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:40.341298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:40.341298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:40.341298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:40.341298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:40.341298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:40.341298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:40.341298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:40.341298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:40.342193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:40.342193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:40.342193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:40.342193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:40.342193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 298/579] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:40.342193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353d51f-9a64-436d-8e1e-f62dea274b12 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:40.344077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 282/580] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:26:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:40.344807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8f624-f8a1-4a47-831d-8d7a85d21c8e None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:40.443514 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353d51f-9a64-436d-8e1e-f62dea274b12 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:40.445715 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8f624-f8a1-4a47-831d-8d7a85d21c8e None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:40.457986 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:40.478180 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d7d225-0645-4447-9323-feb213f84e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:40.478814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7d225-0645-4447-9323-feb213f84e7c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:40.479097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7d225-0645-4447-9323-feb213f84e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:40.479370 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7d225-0645-4447-9323-feb213f84e7c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:40.481304 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c44ebc-f599-494c-b105-da3d547a550a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:40.481947 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c44ebc-f599-494c-b105-da3d547a550a None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:40.482232 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c44ebc-f599-494c-b105-da3d547a550a None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:40.482508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c44ebc-f599-494c-b105-da3d547a550a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:40.483297 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c44ebc-f599-494c-b105-da3d547a550a None admin] RBAC: Adding query filter params (role.id=cc86b64693b6417e90ad9361460574b0, scope.project.id=f37a5f3140ec431da8ea14568654114a, user.id=b1e539006166434790823b839bd745d5) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:40.483605 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c44ebc-f599-494c-b105-da3d547a550a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:40.485020 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c44ebc-f599-494c-b105-da3d547a550a None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:40.489369 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d7d225-0645-4447-9323-feb213f84e7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:40.490802 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 283/581] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:26:40 2022] GET /identity/v3/role_assignments?user.id=b1e539006166434790823b839bd745d5&scope.project.id=f37a5f3140ec431da8ea14568654114a&role.id=cc86b64693b6417e90ad9361460574b0 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:40.491415 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c44ebc-f599-494c-b105-da3d547a550a None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:40.494963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-b3d7d225-0645-4447-9323-feb213f84e7c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:40.494963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:40.494963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:40.494963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:40.494963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:40.494963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:40.494963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:40.494963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:40.494963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:40.494963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:40.494963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:40.494963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:40.494963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:40.494963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:40.494963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:40.494963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:40.495875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:40.495875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:40.495875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:40.495875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:40.495875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:40.495875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:40.495875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:40.495875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:40.495875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:40.495875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:40.495875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:40.495875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:40.495875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:40.495875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:40.495875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:40.495875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:40.496695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:40.496695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:40.496695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:40.496695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:40.496695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:40.496695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:40.496695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:40.496695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:40.496695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:40.496695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:40.496695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:40.496695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:40.496695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 299/582] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Aug 4 04:26:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:40.497385 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d7d225-0645-4447-9323-feb213f84e7c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:40.578538 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:40.622281 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d7d225-0645-4447-9323-feb213f84e7c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:40.624026 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c44ebc-f599-494c-b105-da3d547a550a None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:40.662816 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2e5c5-d270-4be6-9f32-ae4aaac8ddcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:40.663481 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2e5c5-d270-4be6-9f32-ae4aaac8ddcc None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:40.663760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2e5c5-d270-4be6-9f32-ae4aaac8ddcc None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:40.663998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a462b-a9fa-43f9-bc65-436da2c14caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:40.664120 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2e5c5-d270-4be6-9f32-ae4aaac8ddcc None admin] PATH_INFO: `/v3/projects/f37a5f3140ec431da8ea14568654114a/users/caa2901451aa4554bdfcac1bddbc620a/roles/cc86b64693b6417e90ad9361460574b0` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:40.664723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a462b-a9fa-43f9-bc65-436da2c14caa None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:40.665074 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a462b-a9fa-43f9-bc65-436da2c14caa None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:40.665415 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a462b-a9fa-43f9-bc65-436da2c14caa None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:40.666510 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a462b-a9fa-43f9-bc65-436da2c14caa None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:40.666835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a462b-a9fa-43f9-bc65-436da2c14caa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:40.668453 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a462b-a9fa-43f9-bc65-436da2c14caa None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:40.680475 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2e5c5-d270-4be6-9f32-ae4aaac8ddcc None admin] RBAC: Authorizing `identity:create_grant(project_id=f37a5f3140ec431da8ea14568654114a, user_id=caa2901451aa4554bdfcac1bddbc620a, role_id=cc86b64693b6417e90ad9361460574b0)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:40.681958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2e5c5-d270-4be6-9f32-ae4aaac8ddcc None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:40.688802 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 284/583] 10.222.0.47 () {56 vars in 1137 bytes} [Thu Aug 4 04:26:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:40.689574 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a462b-a9fa-43f9-bc65-436da2c14caa None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:40.703261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 300/584] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Aug 4 04:26:40 2022] PUT /identity/v3/projects/f37a5f3140ec431da8ea14568654114a/users/caa2901451aa4554bdfcac1bddbc620a/roles/cc86b64693b6417e90ad9361460574b0 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:26:40.704657 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2e5c5-d270-4be6-9f32-ae4aaac8ddcc None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:40.746167 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:40.805581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a462b-a9fa-43f9-bc65-436da2c14caa None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:40.842100 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:40.848918 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa10128-3303-47a0-b2bc-f6bac84d7f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:40.849973 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa10128-3303-47a0-b2bc-f6bac84d7f0c None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:40.850378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa10128-3303-47a0-b2bc-f6bac84d7f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:40.850747 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa10128-3303-47a0-b2bc-f6bac84d7f0c None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:40.852049 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa10128-3303-47a0-b2bc-f6bac84d7f0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:40.852726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa10128-3303-47a0-b2bc-f6bac84d7f0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:40.854289 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa10128-3303-47a0-b2bc-f6bac84d7f0c None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:40.854798 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2e5c5-d270-4be6-9f32-ae4aaac8ddcc None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:40.870526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 285/585] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:40.872001 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa10128-3303-47a0-b2bc-f6bac84d7f0c None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:40.910436 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc8004-31f8-42bd-bc8a-730401ee7a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:40.911263 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc8004-31f8-42bd-bc8a-730401ee7a3c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:40.911791 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc8004-31f8-42bd-bc8a-730401ee7a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:40.912337 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc8004-31f8-42bd-bc8a-730401ee7a3c None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:40.913615 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc8004-31f8-42bd-bc8a-730401ee7a3c None admin] RBAC: Adding query filter params (name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:40.914034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc8004-31f8-42bd-bc8a-730401ee7a3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:40.915528 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc8004-31f8-42bd-bc8a-730401ee7a3c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:40.926859 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 301/586] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Aug 4 04:26:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:40.927701 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc8004-31f8-42bd-bc8a-730401ee7a3c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:41.031958 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa10128-3303-47a0-b2bc-f6bac84d7f0c None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:41.064444 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:41.077632 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64872d3-b004-4849-921d-fa5c236c8fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:41.078669 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64872d3-b004-4849-921d-fa5c236c8fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:41.079100 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64872d3-b004-4849-921d-fa5c236c8fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:41.079534 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64872d3-b004-4849-921d-fa5c236c8fa1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:41.081535 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc8004-31f8-42bd-bc8a-730401ee7a3c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:41.092470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64872d3-b004-4849-921d-fa5c236c8fa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:41.094160 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64872d3-b004-4849-921d-fa5c236c8fa1 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:41.111481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-a64872d3-b004-4849-921d-fa5c236c8fa1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:41.111481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:41.111481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:41.111481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:41.111481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:41.111481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:41.111481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:41.111481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:41.111481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:41.111481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:41.111481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:41.111481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:41.111481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:41.111481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:41.111481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:41.113663 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:41.113663 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:41.113663 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:41.113663 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:41.113663 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:41.113663 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:41.113663 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:41.113663 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:41.113663 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:41.113663 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:41.113663 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:41.113663 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:41.113663 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:41.113663 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:41.113663 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:41.113663 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:41.113663 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:41.114634 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:41.114634 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:41.114634 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:41.114634 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:41.117512 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 286/587] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:41.118451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de054a5-02fd-444a-ae7e-0324e46ad662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:41.118848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64872d3-b004-4849-921d-fa5c236c8fa1 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:41.119140 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de054a5-02fd-444a-ae7e-0324e46ad662 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:41.119422 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de054a5-02fd-444a-ae7e-0324e46ad662 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:41.119693 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de054a5-02fd-444a-ae7e-0324e46ad662 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:41.128607 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de054a5-02fd-444a-ae7e-0324e46ad662 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:41.130731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de054a5-02fd-444a-ae7e-0324e46ad662 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:41.143065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-1de054a5-02fd-444a-ae7e-0324e46ad662 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 04:26:41.143065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:41.143065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:41.143065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:41.143065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:41.143065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:41.143065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:41.143065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:41.143065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:41.143065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:41.143065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:41.143065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:41.143065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:26:41.143065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:26:41.143065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:26:41.143065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:26:41.144523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:41.144523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:41.144523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:26:41.144523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:41.144523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:26:41.144523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:41.144523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:41.144523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:41.144523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:41.144523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:41.144523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:41.144523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:41.144523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:41.144523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:41.144523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:41.144523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:41.144523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:41.144523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:41.145686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:41.145686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:41.145686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:26:41.145686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:26:41.145686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:26:41.145686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:26:41.145686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:26:41.145686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:26:41.145686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 04:26:41.145686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:41.145686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 302/588] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:41.145686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de054a5-02fd-444a-ae7e-0324e46ad662 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:41.162304 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:41.242859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64872d3-b004-4849-921d-fa5c236c8fa1 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:41.260616 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de054a5-02fd-444a-ae7e-0324e46ad662 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:41.282310 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5c60e-2e33-419d-967c-48e4215190cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:41.284092 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5c60e-2e33-419d-967c-48e4215190cd None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:41.284802 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5c60e-2e33-419d-967c-48e4215190cd None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:41.285360 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5c60e-2e33-419d-967c-48e4215190cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:41.298369 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f5c60e-2e33-419d-967c-48e4215190cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:41.300573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44813c99-322b-4538-a8ca-7aacde0ccb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:41.301192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44813c99-322b-4538-a8ca-7aacde0ccb98 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:41.301473 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44813c99-322b-4538-a8ca-7aacde0ccb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:41.301782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44813c99-322b-4538-a8ca-7aacde0ccb98 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:41.302450 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f5c60e-2e33-419d-967c-48e4215190cd None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:41.302584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44813c99-322b-4538-a8ca-7aacde0ccb98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:41.302750 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44813c99-322b-4538-a8ca-7aacde0ccb98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:41.304282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44813c99-322b-4538-a8ca-7aacde0ccb98 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:41.319198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 303/589] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:41.320341 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44813c99-322b-4538-a8ca-7aacde0ccb98 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:41.321476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-32f5c60e-2e33-419d-967c-48e4215190cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:41.321476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:41.321476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:41.321476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:41.321476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:41.321476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:41.321476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:41.321476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:41.321476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:41.321476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:41.321476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:41.321476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:41.321476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:41.321476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:41.321476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:41.322440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:41.322440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:41.322440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:41.322440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:41.322440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:41.322440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:41.322440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:41.322440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:41.322440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:41.322440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:41.322440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:41.322440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:41.322440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:41.322440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:41.322440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:41.322440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:41.322440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:41.323304 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:41.323304 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:41.323304 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:41.323304 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:41.326411 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 287/590] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:41.327450 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5c60e-2e33-419d-967c-48e4215190cd None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:41.329523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:41.425371 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:41.431336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44813c99-322b-4538-a8ca-7aacde0ccb98 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:41.443869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5c60e-2e33-419d-967c-48e4215190cd None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:41.468913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57b3113-334a-43a6-8de0-a3f382d13cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:41.469581 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b3113-334a-43a6-8de0-a3f382d13cec None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:41.470011 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b3113-334a-43a6-8de0-a3f382d13cec None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:41.470251 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b3113-334a-43a6-8de0-a3f382d13cec None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:41.471694 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57b3113-334a-43a6-8de0-a3f382d13cec None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:41.471843 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57b3113-334a-43a6-8de0-a3f382d13cec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:41.473332 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57b3113-334a-43a6-8de0-a3f382d13cec None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:41.483090 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539b2a6-74ff-4894-bf53-ff8ca3b1836d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:41.484406 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539b2a6-74ff-4894-bf53-ff8ca3b1836d None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:41.484900 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539b2a6-74ff-4894-bf53-ff8ca3b1836d None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:41.485370 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539b2a6-74ff-4894-bf53-ff8ca3b1836d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:41.493207 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 304/591] 10.222.0.47 () {56 vars in 1137 bytes} [Thu Aug 4 04:26:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:41.494248 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57b3113-334a-43a6-8de0-a3f382d13cec None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:41.494598 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539b2a6-74ff-4894-bf53-ff8ca3b1836d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:41.496234 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539b2a6-74ff-4894-bf53-ff8ca3b1836d None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:41.510594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-6539b2a6-74ff-4894-bf53-ff8ca3b1836d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:41.510594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:41.510594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:41.510594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:41.510594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:41.510594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:41.510594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:41.510594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:41.510594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:41.510594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:41.510594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:41.510594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:41.510594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:26:41.510594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:26:41.510594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:26:41.511479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:26:41.511479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:41.511479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:41.511479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:41.511479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:41.511479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:41.511479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:41.511479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:41.511479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:41.511479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:41.511479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:41.511479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:41.511479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:41.511479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:41.511479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:41.511479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:41.511479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:41.512342 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:26:41.512342 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:26:41.512342 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:26:41.512342 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:26:41.512342 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:26:41.512342 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:26:41.512342 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:41.512342 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:41.514982 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 288/592] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:41.516011 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539b2a6-74ff-4894-bf53-ff8ca3b1836d None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:41.561236 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:41.603975 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57b3113-334a-43a6-8de0-a3f382d13cec None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:41.629875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539b2a6-74ff-4894-bf53-ff8ca3b1836d None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:41.641750 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400b2fa-1ea3-4941-9f37-e2299b095577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:41.642324 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400b2fa-1ea3-4941-9f37-e2299b095577 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:41.642603 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400b2fa-1ea3-4941-9f37-e2299b095577 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:41.642952 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400b2fa-1ea3-4941-9f37-e2299b095577 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:41.643748 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6400b2fa-1ea3-4941-9f37-e2299b095577 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:41.644001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6400b2fa-1ea3-4941-9f37-e2299b095577 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:41.645368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6400b2fa-1ea3-4941-9f37-e2299b095577 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:41.665917 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 305/593] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:41.667126 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400b2fa-1ea3-4941-9f37-e2299b095577 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:41.669159 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40136b05-bdbc-496a-960f-efc210fb3d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:41.670265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40136b05-bdbc-496a-960f-efc210fb3d19 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:41.671024 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40136b05-bdbc-496a-960f-efc210fb3d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:41.671572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40136b05-bdbc-496a-960f-efc210fb3d19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:41.685286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40136b05-bdbc-496a-960f-efc210fb3d19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:41.687079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40136b05-bdbc-496a-960f-efc210fb3d19 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:41.696947 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:41.708754 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-40136b05-bdbc-496a-960f-efc210fb3d19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:41.708754 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:41.708754 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:41.708754 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:41.708754 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:41.708754 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:41.708754 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:41.708754 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:41.708754 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:41.708754 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:41.708754 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:41.708754 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:41.708754 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:41.708754 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:41.708754 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:41.709700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:41.709700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:41.709700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:41.709700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:41.709700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:41.709700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:41.709700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:41.709700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:41.709700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:41.709700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:41.709700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:41.709700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:41.709700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:41.709700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:41.709700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:41.709700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:41.709700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:41.710572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:41.710572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:41.710572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:41.710572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:41.713306 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 289/594] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:41.714362 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40136b05-bdbc-496a-960f-efc210fb3d19 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:41.780192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400b2fa-1ea3-4941-9f37-e2299b095577 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:41.805303 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:41.818163 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e0f10-faa4-4b35-a141-eac2df81e72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:41.818792 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e0f10-faa4-4b35-a141-eac2df81e72e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:41.819079 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e0f10-faa4-4b35-a141-eac2df81e72e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:41.819362 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e0f10-faa4-4b35-a141-eac2df81e72e None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:41.820544 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e0f10-faa4-4b35-a141-eac2df81e72e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:41.820544 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e0f10-faa4-4b35-a141-eac2df81e72e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:41.821926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e0f10-faa4-4b35-a141-eac2df81e72e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:41.836143 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 306/595] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:26:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:41.836967 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e0f10-faa4-4b35-a141-eac2df81e72e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:41.866892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40136b05-bdbc-496a-960f-efc210fb3d19 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:41.916565 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3b5f0-e959-47a6-9503-24b2bdb18bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:41.917499 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3b5f0-e959-47a6-9503-24b2bdb18bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:41.917969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3b5f0-e959-47a6-9503-24b2bdb18bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:41.918427 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3b5f0-e959-47a6-9503-24b2bdb18bb5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:41.930691 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b3b5f0-e959-47a6-9503-24b2bdb18bb5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:41.932812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b3b5f0-e959-47a6-9503-24b2bdb18bb5 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:41.947348 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e0f10-faa4-4b35-a141-eac2df81e72e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:41.950258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-46b3b5f0-e959-47a6-9503-24b2bdb18bb5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 04:26:41.950258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:41.950258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:41.950258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:41.950258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:41.950258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:41.950258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:41.950258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:41.950258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:41.950258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:41.950258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:41.950258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:41.950258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:26:41.950258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:26:41.950258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:26:41.950258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:26:41.951191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:41.951191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:41.951191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:26:41.951191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:41.951191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:26:41.951191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:41.951191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:41.951191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:41.951191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:41.951191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:41.951191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:41.951191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:41.951191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:41.951191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:41.951191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:41.951191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:41.951191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:41.951191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:41.952095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:41.952095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:41.952095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:26:41.952095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:26:41.952095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:26:41.952095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:26:41.952095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:26:41.952095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:26:41.952095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 04:26:41.952095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:41.952095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 290/596] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Aug 4 04:26:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:41.952675 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3b5f0-e959-47a6-9503-24b2bdb18bb5 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:41.989671 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1b3e8-9d40-4860-8e20-dba12eb50a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:41.990249 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1b3e8-9d40-4860-8e20-dba12eb50a71 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:41.990537 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1b3e8-9d40-4860-8e20-dba12eb50a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:41.990812 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1b3e8-9d40-4860-8e20-dba12eb50a71 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:41.991510 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1b3e8-9d40-4860-8e20-dba12eb50a71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:41.991901 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1b3e8-9d40-4860-8e20-dba12eb50a71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:41.993201 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1b3e8-9d40-4860-8e20-dba12eb50a71 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:42.010199 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 307/597] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:42.011160 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1b3e8-9d40-4860-8e20-dba12eb50a71 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:42.040796 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:42.067908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:42.088737 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3b5f0-e959-47a6-9503-24b2bdb18bb5 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:42.126605 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac98769-93c1-412e-bbea-e38fd2288811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:42.127967 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac98769-93c1-412e-bbea-e38fd2288811 None admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:42.128342 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac98769-93c1-412e-bbea-e38fd2288811 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:42.128729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac98769-93c1-412e-bbea-e38fd2288811 None admin] PATH_INFO: `/v3/projects/f37a5f3140ec431da8ea14568654114a/users/b77a955866c54a43a7aaa271fefdec57/roles/cc86b64693b6417e90ad9361460574b0` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:42.146323 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac98769-93c1-412e-bbea-e38fd2288811 None admin] RBAC: Authorizing `identity:create_grant(project_id=f37a5f3140ec431da8ea14568654114a, user_id=b77a955866c54a43a7aaa271fefdec57, role_id=cc86b64693b6417e90ad9361460574b0)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:42.146905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1b3e8-9d40-4860-8e20-dba12eb50a71 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:42.147802 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac98769-93c1-412e-bbea-e38fd2288811 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:42.169790 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 291/598] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Aug 4 04:26:41 2022] PUT /identity/v3/projects/f37a5f3140ec431da8ea14568654114a/users/b77a955866c54a43a7aaa271fefdec57/roles/cc86b64693b6417e90ad9361460574b0 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:26:42.171139 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac98769-93c1-412e-bbea-e38fd2288811 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:42.188376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a7877-8f3c-4241-b409-4cd0a1105da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:42.189382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a7877-8f3c-4241-b409-4cd0a1105da6 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:42.190482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a7877-8f3c-4241-b409-4cd0a1105da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:42.191355 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a7877-8f3c-4241-b409-4cd0a1105da6 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:42.192441 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a7877-8f3c-4241-b409-4cd0a1105da6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:42.193007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a7877-8f3c-4241-b409-4cd0a1105da6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:42.194826 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a7877-8f3c-4241-b409-4cd0a1105da6 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:42.220785 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 308/599] 10.222.0.47 () {56 vars in 1139 bytes} [Thu Aug 4 04:26:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:42.226377 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a7877-8f3c-4241-b409-4cd0a1105da6 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:42.356888 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a7877-8f3c-4241-b409-4cd0a1105da6 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:42.362924 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:42.407484 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a0a1f-2f8d-4c1d-acef-7a648e9289f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:42.408560 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a0a1f-2f8d-4c1d-acef-7a648e9289f8 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:42.409360 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a0a1f-2f8d-4c1d-acef-7a648e9289f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:42.409927 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a0a1f-2f8d-4c1d-acef-7a648e9289f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:42.411469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac98769-93c1-412e-bbea-e38fd2288811 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:42.426282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a0a1f-2f8d-4c1d-acef-7a648e9289f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:42.428089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a0a1f-2f8d-4c1d-acef-7a648e9289f8 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:42.429778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:42.448652 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-757a0a1f-2f8d-4c1d-acef-7a648e9289f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:42.448652 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:42.448652 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:42.448652 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:42.448652 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:42.448652 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:42.448652 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:42.448652 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:42.448652 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:42.448652 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:42.448652 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:42.448652 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:42.448652 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:42.448652 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:42.448652 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:42.449606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:42.449606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:42.449606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:42.449606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:42.449606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:42.449606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:42.449606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:42.449606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:42.449606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:42.449606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:42.449606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:42.449606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:42.449606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:42.449606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:42.449606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:42.449606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:42.449606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:42.450482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:42.450482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:42.450482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:42.450482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:42.452654 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 309/600] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:26:42.458541 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a0a1f-2f8d-4c1d-acef-7a648e9289f8 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:42.462247 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9d220-eee5-4062-bed4-82ce56ca828b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:42.462973 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9d220-eee5-4062-bed4-82ce56ca828b None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:42.463258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9d220-eee5-4062-bed4-82ce56ca828b None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:42.464427 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9d220-eee5-4062-bed4-82ce56ca828b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:42.474459 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9d220-eee5-4062-bed4-82ce56ca828b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:42.476602 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9d220-eee5-4062-bed4-82ce56ca828b None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:42.500456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-fbf9d220-eee5-4062-bed4-82ce56ca828b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:42.500456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:42.500456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:42.500456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:42.500456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:42.500456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:42.500456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:42.500456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:42.500456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:42.500456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:42.500456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:42.500456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:42.500456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:26:42.500456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:26:42.500456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:26:42.503581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:26:42.503581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:42.503581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:42.503581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:42.503581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:42.503581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:42.503581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:42.503581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:42.503581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:42.503581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:42.503581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:42.503581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:42.503581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:42.503581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:42.503581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:42.503581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:42.503581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:42.504658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:26:42.504658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:26:42.504658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:26:42.504658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:26:42.504658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:26:42.504658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:26:42.504658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:42.504658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:42.506006 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 292/601] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:42.510794 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9d220-eee5-4062-bed4-82ce56ca828b None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:42.685247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a0a1f-2f8d-4c1d-acef-7a648e9289f8 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:42.708081 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9d220-eee5-4062-bed4-82ce56ca828b None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:42.735357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ddc4f-8e38-40a4-8bf2-3b6bb0c9d678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:42.736216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ddc4f-8e38-40a4-8bf2-3b6bb0c9d678 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:42.737250 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ddc4f-8e38-40a4-8bf2-3b6bb0c9d678 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:42.737812 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ddc4f-8e38-40a4-8bf2-3b6bb0c9d678 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:42.738772 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ddc4f-8e38-40a4-8bf2-3b6bb0c9d678 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:42.739276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ddc4f-8e38-40a4-8bf2-3b6bb0c9d678 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:42.740900 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ddc4f-8e38-40a4-8bf2-3b6bb0c9d678 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:42.754974 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 310/602] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:42.755445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc3913-760d-4305-9b57-eb23e979ad68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:42.756153 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc3913-760d-4305-9b57-eb23e979ad68 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:42.756440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc3913-760d-4305-9b57-eb23e979ad68 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:42.756712 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc3913-760d-4305-9b57-eb23e979ad68 None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:42.757469 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afcb80-400e-46bb-ae72-d469263752fa None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:42.757597 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc3913-760d-4305-9b57-eb23e979ad68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:42.757809 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc3913-760d-4305-9b57-eb23e979ad68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:42.758280 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afcb80-400e-46bb-ae72-d469263752fa None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:42.758740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afcb80-400e-46bb-ae72-d469263752fa None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:42.759470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc3913-760d-4305-9b57-eb23e979ad68 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:42.764864 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 311/603] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:42.766007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afcb80-400e-46bb-ae72-d469263752fa None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:42.774475 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 293/604] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:26:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:42.775756 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae092e5-7413-471f-a3a3-326f3ebd9745 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:42.776066 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae092e5-7413-471f-a3a3-326f3ebd9745 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:42.776910 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae092e5-7413-471f-a3a3-326f3ebd9745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:42.785893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:42.801365 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-5ae092e5-7413-471f-a3a3-326f3ebd9745 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:42.802578 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5ae092e5-7413-471f-a3a3-326f3ebd9745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:42.828524 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:42.885569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 294/605] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:42.885569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae092e5-7413-471f-a3a3-326f3ebd9745 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:42.929958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afcb80-400e-46bb-ae72-d469263752fa None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:42.983417 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53dd9f1-02a0-4d65-8e59-90676914770c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:42.985671 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dd9f1-02a0-4d65-8e59-90676914770c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:42.987339 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dd9f1-02a0-4d65-8e59-90676914770c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:42.988092 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dd9f1-02a0-4d65-8e59-90676914770c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:42.999342 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53dd9f1-02a0-4d65-8e59-90676914770c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:43.000801 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53dd9f1-02a0-4d65-8e59-90676914770c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:43.014944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-e53dd9f1-02a0-4d65-8e59-90676914770c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:43.014944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:43.014944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:43.014944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:43.014944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:43.014944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:43.014944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:43.014944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:43.014944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:43.014944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:43.014944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:43.014944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:43.014944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:26:43.014944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:26:43.014944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:26:43.016964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:26:43.016964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:43.016964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:43.016964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:43.016964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:43.016964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:43.016964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:43.016964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:43.016964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:43.016964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:43.016964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:43.016964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:43.016964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:43.016964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:43.016964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:43.016964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:43.016964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:43.018777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:26:43.018777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:26:43.018777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:26:43.018777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:26:43.018777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:26:43.018777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:26:43.018777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:43.018777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:43.018777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 312/606] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:43.018777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53dd9f1-02a0-4d65-8e59-90676914770c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:43.058406 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae092e5-7413-471f-a3a3-326f3ebd9745 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:43.099515 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f9bee-040c-456b-932e-6b4d4935a5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:43.099515 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f9bee-040c-456b-932e-6b4d4935a5e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:43.099792 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f9bee-040c-456b-932e-6b4d4935a5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:43.099892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f9bee-040c-456b-932e-6b4d4935a5e9 None admin] PATH_INFO: `/v3/projects/f37a5f3140ec431da8ea14568654114a/users/8b1d67b28dca49ca9cb0ff28cad62aa1/roles/cc86b64693b6417e90ad9361460574b0` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:43.110772 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:43.117322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f9bee-040c-456b-932e-6b4d4935a5e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=f37a5f3140ec431da8ea14568654114a, user_id=8b1d67b28dca49ca9cb0ff28cad62aa1, role_id=cc86b64693b6417e90ad9361460574b0)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:43.119139 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f9bee-040c-456b-932e-6b4d4935a5e9 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:43.135701 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 295/607] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Aug 4 04:26:42 2022] PUT /identity/v3/projects/f37a5f3140ec431da8ea14568654114a/users/8b1d67b28dca49ca9cb0ff28cad62aa1/roles/cc86b64693b6417e90ad9361460574b0 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:26:43.136479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f9bee-040c-456b-932e-6b4d4935a5e9 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:43.150527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53dd9f1-02a0-4d65-8e59-90676914770c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:43.155851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:43.195650 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9538ee2-174b-42a3-8ee2-19fb9d41b235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:43.196513 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9538ee2-174b-42a3-8ee2-19fb9d41b235 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:43.196898 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9538ee2-174b-42a3-8ee2-19fb9d41b235 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:43.197276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9538ee2-174b-42a3-8ee2-19fb9d41b235 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:43.211889 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9538ee2-174b-42a3-8ee2-19fb9d41b235 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:43.219317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-b9538ee2-174b-42a3-8ee2-19fb9d41b235 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:43.219317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:43.219317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:43.219317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:43.219317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:43.219317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:43.219317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:43.219317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:43.219317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:43.219317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:43.219317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:43.219317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:43.219317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:43.219317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:43.219317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:43.219317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:43.220522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:43.220522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:43.220522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:43.220522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:43.220522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:43.220522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:43.220522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:43.220522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:43.220522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:43.220522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:43.220522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:43.220522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:43.220522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:43.220522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:43.220522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:43.220522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:43.221483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:43.221483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:43.221483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:43.221483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:43.221483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:43.221483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:43.221483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:43.221483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:43.221483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:43.221483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:43.221483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:43.221483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:43.226280 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 313/608] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Aug 4 04:26:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:43.227908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b0343-c221-4b83-89e5-a8c3d84e2463 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:43.228280 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b0343-c221-4b83-89e5-a8c3d84e2463 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:43.228713 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b0343-c221-4b83-89e5-a8c3d84e2463 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:43.234335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 314/609] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:43.234335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b0343-c221-4b83-89e5-a8c3d84e2463 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:43.299691 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f9bee-040c-456b-932e-6b4d4935a5e9 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:43.345069 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa31e0f-a491-46b3-96ed-bbacf6bb973c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:43.346233 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa31e0f-a491-46b3-96ed-bbacf6bb973c None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:43.346749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa31e0f-a491-46b3-96ed-bbacf6bb973c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:43.347061 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa31e0f-a491-46b3-96ed-bbacf6bb973c None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:43.349639 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa31e0f-a491-46b3-96ed-bbacf6bb973c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:43.350150 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa31e0f-a491-46b3-96ed-bbacf6bb973c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:43.352179 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa31e0f-a491-46b3-96ed-bbacf6bb973c None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:43.359578 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b0343-c221-4b83-89e5-a8c3d84e2463 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:43.372171 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 296/610] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:26:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:43.372171 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1868a91-afea-4cc0-aa62-3b936db42130 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:43.372171 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1868a91-afea-4cc0-aa62-3b936db42130 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:43.372171 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1868a91-afea-4cc0-aa62-3b936db42130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:43.398108 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-f1868a91-afea-4cc0-aa62-3b936db42130 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:43.400465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-f1868a91-afea-4cc0-aa62-3b936db42130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:43.404346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d78a2-78d4-4a58-909f-24e5c3d074f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:43.405461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d78a2-78d4-4a58-909f-24e5c3d074f4 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:43.405461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d78a2-78d4-4a58-909f-24e5c3d074f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:43.405912 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d78a2-78d4-4a58-909f-24e5c3d074f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:43.406756 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d78a2-78d4-4a58-909f-24e5c3d074f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:43.407168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d78a2-78d4-4a58-909f-24e5c3d074f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:43.408711 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d78a2-78d4-4a58-909f-24e5c3d074f4 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:43.410623 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:43.425159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 315/611] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Aug 4 04:26:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:43.426286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d78a2-78d4-4a58-909f-24e5c3d074f4 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:43.490919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:43.505787 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 297/612] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:43.505787 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1868a91-afea-4cc0-aa62-3b936db42130 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:43.575720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d78a2-78d4-4a58-909f-24e5c3d074f4 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:43.630190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e239b-42fe-4da3-ab0f-5786733bed60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:43.631263 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e239b-42fe-4da3-ab0f-5786733bed60 None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:43.631764 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e239b-42fe-4da3-ab0f-5786733bed60 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:43.632328 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e239b-42fe-4da3-ab0f-5786733bed60 None admin] PATH_INFO: `/v3/projects/f37a5f3140ec431da8ea14568654114a/users/1a2a647f97fc4d369f562e8b5a4bf68c/roles/cc86b64693b6417e90ad9361460574b0` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:43.642411 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1868a91-afea-4cc0-aa62-3b936db42130 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:43.655938 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e239b-42fe-4da3-ab0f-5786733bed60 None admin] RBAC: Authorizing `identity:create_grant(project_id=f37a5f3140ec431da8ea14568654114a, user_id=1a2a647f97fc4d369f562e8b5a4bf68c, role_id=cc86b64693b6417e90ad9361460574b0)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:43.657787 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e239b-42fe-4da3-ab0f-5786733bed60 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:43.681352 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 316/613] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Aug 4 04:26:43 2022] PUT /identity/v3/projects/f37a5f3140ec431da8ea14568654114a/users/1a2a647f97fc4d369f562e8b5a4bf68c/roles/cc86b64693b6417e90ad9361460574b0 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:26:43.682937 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e239b-42fe-4da3-ab0f-5786733bed60 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:43.685422 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a23ca-ef54-471f-904b-7a712680e11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:43.686732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a23ca-ef54-471f-904b-7a712680e11f None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:43.687028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a23ca-ef54-471f-904b-7a712680e11f None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:43.687301 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a23ca-ef54-471f-904b-7a712680e11f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:43.704091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a23ca-ef54-471f-904b-7a712680e11f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:43.705772 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a23ca-ef54-471f-904b-7a712680e11f None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:43.731383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-a65a23ca-ef54-471f-904b-7a712680e11f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:43.731383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:43.731383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:43.731383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:43.731383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:43.731383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:43.731383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:43.731383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:43.731383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:43.731383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:43.731383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:43.731383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:43.731383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:43.731383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:43.731383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:43.732526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:43.732526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:43.732526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:43.732526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:43.732526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:43.732526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:43.732526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:43.732526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:43.732526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:43.732526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:43.732526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:43.732526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:43.732526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:43.732526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:43.732526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:43.732526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:43.732526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:43.733943 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:43.733943 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:43.733943 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:43.733943 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:43.733943 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 298/614] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:43.740910 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a23ca-ef54-471f-904b-7a712680e11f None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:43.766760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:43.809744 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:43.849335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e239b-42fe-4da3-ab0f-5786733bed60 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:43.897869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d4d2e-6b17-4af0-bdbe-97e782f3ae42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:43.900168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d4d2e-6b17-4af0-bdbe-97e782f3ae42 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:43.904123 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d4d2e-6b17-4af0-bdbe-97e782f3ae42 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:43.904749 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d4d2e-6b17-4af0-bdbe-97e782f3ae42 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:43.930402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d4d2e-6b17-4af0-bdbe-97e782f3ae42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:43.937886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-780d4d2e-6b17-4af0-bdbe-97e782f3ae42 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:43.937886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:43.937886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:43.937886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:43.937886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:43.937886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:43.937886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:43.937886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:43.937886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:43.937886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:43.937886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:43.937886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:43.937886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:43.937886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:43.937886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:43.937886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:43.938867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:43.938867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:43.938867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:43.938867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:43.938867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:43.938867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:43.938867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:43.938867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:43.938867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:43.938867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:43.938867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:43.938867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:43.938867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:43.938867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:43.938867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:43.938867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:43.940270 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:43.940270 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:43.940270 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:43.940270 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:43.940270 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:43.940270 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:43.940270 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:43.940270 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:43.940270 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:43.940270 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:43.940270 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:43.940270 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:43.940270 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 317/615] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Aug 4 04:26:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:43.944884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a23ca-ef54-471f-904b-7a712680e11f None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:43.945458 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d4d2e-6b17-4af0-bdbe-97e782f3ae42 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:43.996472 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f52ef-f683-4407-bbdf-6d0ecf3b7531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:43.997161 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f52ef-f683-4407-bbdf-6d0ecf3b7531 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:43.997444 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f52ef-f683-4407-bbdf-6d0ecf3b7531 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:43.997744 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f52ef-f683-4407-bbdf-6d0ecf3b7531 None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:43.998458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f52ef-f683-4407-bbdf-6d0ecf3b7531 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:43.998768 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f52ef-f683-4407-bbdf-6d0ecf3b7531 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:44.000168 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f52ef-f683-4407-bbdf-6d0ecf3b7531 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:44.024207 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 299/616] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:44.028713 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f52ef-f683-4407-bbdf-6d0ecf3b7531 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:44.118144 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d4d2e-6b17-4af0-bdbe-97e782f3ae42 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:44.145946 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:44.193099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f52ef-f683-4407-bbdf-6d0ecf3b7531 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:44.210140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:44.223083 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09fcf44-9301-4855-a68b-85adce8001c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:44.224121 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fcf44-9301-4855-a68b-85adce8001c4 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:44.224600 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fcf44-9301-4855-a68b-85adce8001c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:44.225102 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fcf44-9301-4855-a68b-85adce8001c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:44.226085 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09fcf44-9301-4855-a68b-85adce8001c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:44.226593 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09fcf44-9301-4855-a68b-85adce8001c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:44.228174 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09fcf44-9301-4855-a68b-85adce8001c4 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:44.245584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2a9fc-34b7-43ee-bf9a-3386832d6a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:44.245584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2a9fc-34b7-43ee-bf9a-3386832d6a8c None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:44.245584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2a9fc-34b7-43ee-bf9a-3386832d6a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:44.245584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2a9fc-34b7-43ee-bf9a-3386832d6a8c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:44.249473 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 318/617] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Aug 4 04:26:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:44.255344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09fcf44-9301-4855-a68b-85adce8001c4 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:44.258838 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2a9fc-34b7-43ee-bf9a-3386832d6a8c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:44.261416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2a9fc-34b7-43ee-bf9a-3386832d6a8c None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:44.274005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-b2c2a9fc-34b7-43ee-bf9a-3386832d6a8c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 04:26:44.274005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:44.274005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:44.274005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:44.274005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:44.274005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:44.274005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:44.274005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:44.274005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:44.274005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:44.274005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:44.274005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:44.274005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:26:44.274005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:26:44.274005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:26:44.274005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:26:44.274965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:44.274965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:44.274965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:26:44.274965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:44.274965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:26:44.274965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:44.274965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:44.274965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:44.274965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:44.274965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:44.274965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:44.274965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:44.274965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:44.274965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:44.274965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:44.274965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:44.274965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:44.274965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:44.275853 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:44.275853 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:44.275853 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:26:44.275853 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:26:44.275853 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:26:44.275853 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:26:44.275853 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:26:44.275853 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:26:44.275853 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 04:26:44.275853 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:44.275853 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 300/618] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:26:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:44.286574 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2a9fc-34b7-43ee-bf9a-3386832d6a8c None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:44.396469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2a9fc-34b7-43ee-bf9a-3386832d6a8c None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:44.435693 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15889946-fc0f-4a61-972f-5a4a24e76244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:44.436325 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15889946-fc0f-4a61-972f-5a4a24e76244 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:44.436609 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15889946-fc0f-4a61-972f-5a4a24e76244 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:44.436877 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15889946-fc0f-4a61-972f-5a4a24e76244 None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:44.438059 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15889946-fc0f-4a61-972f-5a4a24e76244 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:44.438323 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15889946-fc0f-4a61-972f-5a4a24e76244 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:44.440089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15889946-fc0f-4a61-972f-5a4a24e76244 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:44.468696 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 301/619] 10.222.0.47 () {56 vars in 1135 bytes} [Thu Aug 4 04:26:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:44.474307 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09fcf44-9301-4855-a68b-85adce8001c4 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:44.477192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15889946-fc0f-4a61-972f-5a4a24e76244 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:44.535376 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:44.550629 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efded6-4bb0-47c6-b6e9-c31e85809e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:44.551475 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efded6-4bb0-47c6-b6e9-c31e85809e5e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:44.551802 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efded6-4bb0-47c6-b6e9-c31e85809e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:44.552084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efded6-4bb0-47c6-b6e9-c31e85809e5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:44.571337 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4efded6-4bb0-47c6-b6e9-c31e85809e5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:44.572811 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4efded6-4bb0-47c6-b6e9-c31e85809e5e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:44.604221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-d4efded6-4bb0-47c6-b6e9-c31e85809e5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:44.604221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:44.604221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:44.604221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:44.604221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:44.604221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:44.604221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:44.604221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:44.604221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:44.604221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:44.604221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:44.604221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:44.604221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:44.604221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:44.604221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:44.605402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:44.605402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:44.605402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:44.605402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:44.605402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:44.605402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:44.605402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:44.605402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:44.605402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:44.605402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:44.605402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:44.605402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:44.605402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:44.605402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:44.605402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:44.605402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:44.605402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:44.606368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:44.606368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:44.606368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:44.606368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:44.607952 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 319/620] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:26:44.614145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efded6-4bb0-47c6-b6e9-c31e85809e5e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:44.658101 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:44.714385 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15889946-fc0f-4a61-972f-5a4a24e76244 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:44.749876 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efded6-4bb0-47c6-b6e9-c31e85809e5e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:44.752823 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40feadf-8518-4da3-adc1-5f259cc63efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:44.753927 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40feadf-8518-4da3-adc1-5f259cc63efe None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:44.754216 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40feadf-8518-4da3-adc1-5f259cc63efe None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:44.754492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40feadf-8518-4da3-adc1-5f259cc63efe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:44.766091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40feadf-8518-4da3-adc1-5f259cc63efe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:44.768357 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40feadf-8518-4da3-adc1-5f259cc63efe None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:44.788658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d12fe-a758-41f4-9dc2-9a4405b9d7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:44.789278 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d12fe-a758-41f4-9dc2-9a4405b9d7a7 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:44.789572 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d12fe-a758-41f4-9dc2-9a4405b9d7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:44.789864 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d12fe-a758-41f4-9dc2-9a4405b9d7a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:44.790568 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d12fe-a758-41f4-9dc2-9a4405b9d7a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:44.790877 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d12fe-a758-41f4-9dc2-9a4405b9d7a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:44.792248 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d12fe-a758-41f4-9dc2-9a4405b9d7a7 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:44.796439 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-a40feadf-8518-4da3-adc1-5f259cc63efe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:44.796439 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:44.796439 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:44.796439 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:44.796439 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:44.796439 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:44.796439 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:44.796439 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:44.796439 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:44.796439 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:44.796439 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:44.796439 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:44.796439 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:44.796439 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:44.796439 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:44.797286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:44.797286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:44.797286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:44.797286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:44.797286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:44.797286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:44.797286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:44.797286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:44.797286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:44.797286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:44.797286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:44.797286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:44.797286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:44.797286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:44.797286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:44.797286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:44.797286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:44.798513 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:44.798513 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:44.798513 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:44.798513 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:44.798513 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 302/621] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:44.801026 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed079db2-4a07-4afc-9449-2f82b5b33925 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:44.801628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed079db2-4a07-4afc-9449-2f82b5b33925 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:44.801872 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed079db2-4a07-4afc-9449-2f82b5b33925 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:44.809754 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 303/623] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:44 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:44.811232 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed079db2-4a07-4afc-9449-2f82b5b33925 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:44.813580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 320/623] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:44.815541 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf536f5-755f-4cb9-b144-db3a9ca51d73 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:44.815831 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf536f5-755f-4cb9-b144-db3a9ca51d73 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:44.816232 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf536f5-755f-4cb9-b144-db3a9ca51d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:44.835523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-bbf536f5-755f-4cb9-b144-db3a9ca51d73 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:44.836784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-bbf536f5-755f-4cb9-b144-db3a9ca51d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:44.915921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:44.925644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 321/624] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:44.925644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf536f5-755f-4cb9-b144-db3a9ca51d73 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:44.939442 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:44.963435 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed079db2-4a07-4afc-9449-2f82b5b33925 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:45.004406 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed716da9-8249-461f-b156-d9b7658da2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:45.005026 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed716da9-8249-461f-b156-d9b7658da2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:45.005307 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed716da9-8249-461f-b156-d9b7658da2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:45.005616 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed716da9-8249-461f-b156-d9b7658da2c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:45.006316 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed716da9-8249-461f-b156-d9b7658da2c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:45.006622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed716da9-8249-461f-b156-d9b7658da2c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:45.007992 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed716da9-8249-461f-b156-d9b7658da2c5 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:45.022176 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 304/625] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:45.023039 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed716da9-8249-461f-b156-d9b7658da2c5 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:45.058197 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf536f5-755f-4cb9-b144-db3a9ca51d73 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:45.105389 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8d511-cf7d-4070-a162-153effc49150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:45.106343 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8d511-cf7d-4070-a162-153effc49150 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:45.106783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8d511-cf7d-4070-a162-153effc49150 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:45.107212 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8d511-cf7d-4070-a162-153effc49150 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:45.119489 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e8d511-cf7d-4070-a162-153effc49150 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:45.121185 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e8d511-cf7d-4070-a162-153effc49150 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:45.135266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-49e8d511-cf7d-4070-a162-153effc49150 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 04:26:45.135266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:45.135266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:45.135266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:45.135266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:45.135266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:45.135266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:45.135266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:45.135266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:45.135266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:45.135266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:45.135266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:45.135266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:26:45.135266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:26:45.135266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:26:45.135266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:26:45.136261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:45.136261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:45.136261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:26:45.136261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:45.136261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:26:45.136261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:45.136261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:45.136261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:45.136261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:45.136261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:45.136261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:45.136261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:45.136261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:45.136261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:45.136261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:45.136261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:45.136261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:45.136261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:45.137198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:45.137198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:45.137198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:26:45.137198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:26:45.137198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:26:45.137198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:26:45.137198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:26:45.137198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:26:45.137198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 04:26:45.137198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:45.137198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 322/626] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Aug 4 04:26:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:45.137198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8d511-cf7d-4070-a162-153effc49150 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:45.143292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed716da9-8249-461f-b156-d9b7658da2c5 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:45.179562 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9121dc-6fdf-4a21-91ee-706cd13116fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:45.180136 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9121dc-6fdf-4a21-91ee-706cd13116fc None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:45.180418 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9121dc-6fdf-4a21-91ee-706cd13116fc None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:45.180692 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9121dc-6fdf-4a21-91ee-706cd13116fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:45.190705 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:45.193933 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9121dc-6fdf-4a21-91ee-706cd13116fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:45.199998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-5b9121dc-6fdf-4a21-91ee-706cd13116fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:45.199998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:45.199998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:45.199998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:45.199998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:45.199998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:45.199998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:45.199998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:45.199998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:45.199998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:45.199998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:45.199998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:45.199998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:45.199998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:45.199998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:45.199998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:45.201140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:45.201140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:45.201140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:45.201140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:45.201140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:45.201140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:45.201140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:45.201140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:45.201140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:45.201140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:45.201140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:45.201140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:45.201140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:45.201140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:45.201140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:45.201140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:45.202226 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:45.202226 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:45.202226 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:45.202226 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:45.202226 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:45.202226 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:45.202226 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:45.202226 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:45.202226 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:45.202226 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:45.202226 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:45.202226 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:45.209820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 305/627] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Aug 4 04:26:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:45.210932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9121dc-6fdf-4a21-91ee-706cd13116fc None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:45.227445 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:45.275985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8d511-cf7d-4070-a162-153effc49150 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:45.323399 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60a0c1-1eb2-4bc2-bf69-771a5eecf2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:45.324642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60a0c1-1eb2-4bc2-bf69-771a5eecf2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:45.325120 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60a0c1-1eb2-4bc2-bf69-771a5eecf2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:45.325841 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60a0c1-1eb2-4bc2-bf69-771a5eecf2f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:45.331820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9121dc-6fdf-4a21-91ee-706cd13116fc None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:45.337099 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60a0c1-1eb2-4bc2-bf69-771a5eecf2f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:45.338814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60a0c1-1eb2-4bc2-bf69-771a5eecf2f0 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:45.354539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-2d60a0c1-1eb2-4bc2-bf69-771a5eecf2f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:45.354539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:45.354539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:45.354539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:45.354539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:45.354539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:45.354539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:45.354539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:45.354539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:45.354539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:45.354539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:45.354539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:45.354539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:26:45.354539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:26:45.354539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:26:45.355479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:26:45.355479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:45.355479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:45.355479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:45.355479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:45.355479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:45.355479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:45.355479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:45.355479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:45.355479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:45.355479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:45.355479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:45.355479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:45.355479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:45.355479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:45.355479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:45.355479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:45.356344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:26:45.356344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:26:45.356344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:26:45.356344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:26:45.356344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:26:45.356344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:26:45.356344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:45.356344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:45.358419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 323/628] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:45.359599 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60a0c1-1eb2-4bc2-bf69-771a5eecf2f0 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:45.379313 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4d88d-ff2a-45b6-a681-53913c64a60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:45.380469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4d88d-ff2a-45b6-a681-53913c64a60f None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:45.380901 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4d88d-ff2a-45b6-a681-53913c64a60f None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:45.381339 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4d88d-ff2a-45b6-a681-53913c64a60f None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:45.382882 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4d88d-ff2a-45b6-a681-53913c64a60f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:45.383364 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4d88d-ff2a-45b6-a681-53913c64a60f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:45.384935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4d88d-ff2a-45b6-a681-53913c64a60f None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:45.410587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 306/629] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Aug 4 04:26:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:45.411682 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4d88d-ff2a-45b6-a681-53913c64a60f None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:45.491456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:45.492001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60a0c1-1eb2-4bc2-bf69-771a5eecf2f0 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:45.535800 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:45.543907 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c62b9a-a2cc-4251-8fa0-28de923f8394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:45.544953 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c62b9a-a2cc-4251-8fa0-28de923f8394 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:45.545828 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c62b9a-a2cc-4251-8fa0-28de923f8394 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:45.546260 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c62b9a-a2cc-4251-8fa0-28de923f8394 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:45.546985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c62b9a-a2cc-4251-8fa0-28de923f8394 None admin] RBAC: Adding query filter params (name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:45.547342 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c62b9a-a2cc-4251-8fa0-28de923f8394 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:45.548588 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4d88d-ff2a-45b6-a681-53913c64a60f None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:45.549085 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c62b9a-a2cc-4251-8fa0-28de923f8394 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:45.561935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 324/630] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Aug 4 04:26:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:45.561935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c62b9a-a2cc-4251-8fa0-28de923f8394 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:45.600578 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a448032-087f-4314-981e-e39ade8bb3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:45.601306 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a448032-087f-4314-981e-e39ade8bb3fe None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:45.601601 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a448032-087f-4314-981e-e39ade8bb3fe None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:45.601869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a448032-087f-4314-981e-e39ade8bb3fe None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:45.602615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a448032-087f-4314-981e-e39ade8bb3fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:45.602919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a448032-087f-4314-981e-e39ade8bb3fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:45.604326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a448032-087f-4314-981e-e39ade8bb3fe None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:45.621032 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 307/631] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:26:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:45.621032 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20ee99-7d33-4fcf-9dce-6ffac65107e7 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:45.621032 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20ee99-7d33-4fcf-9dce-6ffac65107e7 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:45.621032 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20ee99-7d33-4fcf-9dce-6ffac65107e7 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:45.621032 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 308/632] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:45.621032 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20ee99-7d33-4fcf-9dce-6ffac65107e7 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:45.675879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c62b9a-a2cc-4251-8fa0-28de923f8394 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:45.711639 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074e55f-10b0-4d62-9771-9e13d8aea125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:45.712371 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074e55f-10b0-4d62-9771-9e13d8aea125 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:45.712691 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074e55f-10b0-4d62-9771-9e13d8aea125 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:45.713079 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074e55f-10b0-4d62-9771-9e13d8aea125 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:45.730614 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20ee99-7d33-4fcf-9dce-6ffac65107e7 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:45.748067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:45.766966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df78d7d-2e41-47ff-bfbc-d4fe63297361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:45.768231 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df78d7d-2e41-47ff-bfbc-d4fe63297361 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:45.768518 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df78d7d-2e41-47ff-bfbc-d4fe63297361 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:45.768788 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df78d7d-2e41-47ff-bfbc-d4fe63297361 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:45.779852 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df78d7d-2e41-47ff-bfbc-d4fe63297361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:45.780666 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:45.781725 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df78d7d-2e41-47ff-bfbc-d4fe63297361 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:45.792452 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074e55f-10b0-4d62-9771-9e13d8aea125 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:45.793976 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074e55f-10b0-4d62-9771-9e13d8aea125 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:45.798110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-8df78d7d-2e41-47ff-bfbc-d4fe63297361 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:45.798110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:45.798110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:45.798110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:45.798110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:45.798110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:45.798110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:45.798110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:45.798110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:45.798110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:45.798110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:45.798110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:45.798110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:45.798110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:45.798110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:45.798962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:45.798962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:45.798962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:45.798962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:45.798962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:45.798962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:45.798962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:45.798962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:45.798962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:45.798962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:45.798962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:45.798962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:45.798962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:45.798962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:45.798962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:45.798962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:45.798962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:45.800561 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:45.800561 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:45.800561 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:45.800561 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:45.800561 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 309/633] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:45.800561 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df78d7d-2e41-47ff-bfbc-d4fe63297361 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:45.923884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 325/634] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:26:45.925865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405aecc-dee6-42e1-9aba-e142e5bd728d None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:45.926227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405aecc-dee6-42e1-9aba-e142e5bd728d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:45.926566 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405aecc-dee6-42e1-9aba-e142e5bd728d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:45.930657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df78d7d-2e41-47ff-bfbc-d4fe63297361 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:45.944679 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-d405aecc-dee6-42e1-9aba-e142e5bd728d None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:45.946201 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d405aecc-dee6-42e1-9aba-e142e5bd728d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:45.971994 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e0b16-c2d1-473b-9442-d24206ae1bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:45.972706 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e0b16-c2d1-473b-9442-d24206ae1bb6 None admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:45.972985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e0b16-c2d1-473b-9442-d24206ae1bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:45.973253 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e0b16-c2d1-473b-9442-d24206ae1bb6 None admin] PATH_INFO: `/v3/projects/f37a5f3140ec431da8ea14568654114a/users/b1e539006166434790823b839bd745d5/roles/cc86b64693b6417e90ad9361460574b0` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:45.988258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e0b16-c2d1-473b-9442-d24206ae1bb6 None admin] RBAC: Authorizing `identity:create_grant(project_id=f37a5f3140ec431da8ea14568654114a, user_id=b1e539006166434790823b839bd745d5, role_id=cc86b64693b6417e90ad9361460574b0)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:45.989712 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e0b16-c2d1-473b-9442-d24206ae1bb6 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:46.009658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 310/635] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Aug 4 04:26:45 2022] PUT /identity/v3/projects/f37a5f3140ec431da8ea14568654114a/users/b1e539006166434790823b839bd745d5/roles/cc86b64693b6417e90ad9361460574b0 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:26:46.009658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e0b16-c2d1-473b-9442-d24206ae1bb6 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:46.031903 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 326/636] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:46.032842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405aecc-dee6-42e1-9aba-e142e5bd728d None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:46.035659 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:46.079670 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:46.141856 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e0b16-c2d1-473b-9442-d24206ae1bb6 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:46.169191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405aecc-dee6-42e1-9aba-e142e5bd728d None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:46.187352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508a887-4ca8-4f09-a051-7e21997beb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:46.188145 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508a887-4ca8-4f09-a051-7e21997beb0f None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:46.188426 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508a887-4ca8-4f09-a051-7e21997beb0f None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:46.188696 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508a887-4ca8-4f09-a051-7e21997beb0f None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:46.190665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508a887-4ca8-4f09-a051-7e21997beb0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:46.191128 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508a887-4ca8-4f09-a051-7e21997beb0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:46.192584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508a887-4ca8-4f09-a051-7e21997beb0f None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:46.206306 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 311/637] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:46.207283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508a887-4ca8-4f09-a051-7e21997beb0f None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:46.217115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733414d9-6642-40dd-92b6-b7c9b7687910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:46.217837 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733414d9-6642-40dd-92b6-b7c9b7687910 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:46.218106 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733414d9-6642-40dd-92b6-b7c9b7687910 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:46.218380 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733414d9-6642-40dd-92b6-b7c9b7687910 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:46.229878 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733414d9-6642-40dd-92b6-b7c9b7687910 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:46.231356 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733414d9-6642-40dd-92b6-b7c9b7687910 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:46.250739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-733414d9-6642-40dd-92b6-b7c9b7687910 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:46.250739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:46.250739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:46.250739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:46.250739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:46.250739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:46.250739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:46.250739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:46.250739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:46.250739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:46.250739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:46.250739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:46.250739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:46.250739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:46.250739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:46.251738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:46.251738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:46.251738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:46.251738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:46.251738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:46.251738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:46.251738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:46.251738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:46.251738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:46.251738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:46.251738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:46.251738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:46.251738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:46.251738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:46.251738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:46.251738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:46.251738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:46.254227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:46.254227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:46.254227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:46.254227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:46.254227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 327/638] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:46.254877 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a7f33-a9b7-4d97-9b6e-33d3a5b8c612 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:46.255138 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a7f33-a9b7-4d97-9b6e-33d3a5b8c612 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:46.255413 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a7f33-a9b7-4d97-9b6e-33d3a5b8c612 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:46.257406 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 328/639] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:46.261723 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a7f33-a9b7-4d97-9b6e-33d3a5b8c612 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:46.300060 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:46.323521 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508a887-4ca8-4f09-a051-7e21997beb0f None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:46.347176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:46.362208 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9400c1e-eef0-487d-8bd6-d7da25a9f8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:46.362912 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9400c1e-eef0-487d-8bd6-d7da25a9f8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:46.363200 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9400c1e-eef0-487d-8bd6-d7da25a9f8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:46.363473 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9400c1e-eef0-487d-8bd6-d7da25a9f8e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:46.374293 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9400c1e-eef0-487d-8bd6-d7da25a9f8e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:46.379543 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a7f33-a9b7-4d97-9b6e-33d3a5b8c612 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:46.380613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-f9400c1e-eef0-487d-8bd6-d7da25a9f8e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:46.380613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:46.380613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:46.380613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:46.380613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:46.380613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:46.380613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:46.380613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:46.380613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:46.380613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:46.380613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:46.380613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:46.380613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:46.380613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:46.380613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:46.380613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:46.381505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:46.381505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:46.381505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:46.381505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:46.381505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:46.381505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:46.381505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:46.381505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:46.381505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:46.381505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:46.381505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:46.381505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:46.381505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:46.381505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:46.381505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:46.381505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:46.382405 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:46.382405 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:46.382405 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:46.382405 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:46.382405 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:46.382405 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:46.382405 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:46.382405 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:46.382405 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:46.382405 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:46.382405 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:46.382405 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:46.382405 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 312/640] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Aug 4 04:26:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:46.383718 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9400c1e-eef0-487d-8bd6-d7da25a9f8e3 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:46.422066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11deb330-6733-4b11-ab19-82d7e24be231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:46.422872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11deb330-6733-4b11-ab19-82d7e24be231 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:46.423257 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11deb330-6733-4b11-ab19-82d7e24be231 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:46.423632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11deb330-6733-4b11-ab19-82d7e24be231 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:46.432629 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11deb330-6733-4b11-ab19-82d7e24be231 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:46.434217 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11deb330-6733-4b11-ab19-82d7e24be231 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:46.445809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-11deb330-6733-4b11-ab19-82d7e24be231 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 04:26:46.445809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:46.445809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:46.445809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:46.445809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:46.445809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:46.445809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:46.445809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:46.445809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:46.445809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:46.445809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:46.445809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:46.445809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:26:46.445809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:26:46.445809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:26:46.445809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:26:46.447028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:46.447028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:46.447028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:26:46.447028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:46.447028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:26:46.447028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:46.447028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:46.447028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:46.447028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:46.447028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:46.447028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:46.447028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:46.447028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:46.447028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:46.447028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:46.447028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:46.447028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:46.447028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:46.448154 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:46.448154 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:46.448154 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:26:46.448154 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:26:46.448154 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:26:46.448154 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:26:46.448154 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:26:46.448154 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:26:46.448154 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 04:26:46.448154 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:46.454096 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 329/641] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:46.454096 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da6e8d-31ce-4dbe-b145-1323413f74b1 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:46.454096 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da6e8d-31ce-4dbe-b145-1323413f74b1 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:46.454096 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da6e8d-31ce-4dbe-b145-1323413f74b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:46.472543 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-a0da6e8d-31ce-4dbe-b145-1323413f74b1 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:46.473783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a0da6e8d-31ce-4dbe-b145-1323413f74b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:46.502333 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9400c1e-eef0-487d-8bd6-d7da25a9f8e3 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:46.536707 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac887dd-86c9-476d-8485-714e40216120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:46.537416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac887dd-86c9-476d-8485-714e40216120 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:46.537723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac887dd-86c9-476d-8485-714e40216120 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:46.537996 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac887dd-86c9-476d-8485-714e40216120 None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:46.538685 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac887dd-86c9-476d-8485-714e40216120 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:46.538985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac887dd-86c9-476d-8485-714e40216120 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:46.540327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac887dd-86c9-476d-8485-714e40216120 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:46.550153 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:46.552819 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 330/642] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:46.553342 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 313/643] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:46.554297 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac887dd-86c9-476d-8485-714e40216120 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:46.565736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da6e8d-31ce-4dbe-b145-1323413f74b1 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:46.631236 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:46.666322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac887dd-86c9-476d-8485-714e40216120 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:46.676005 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da6e8d-31ce-4dbe-b145-1323413f74b1 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:46.701753 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c0628-1879-47ee-98a2-a1b7a8ee76db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:46.702364 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c0628-1879-47ee-98a2-a1b7a8ee76db None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:46.702641 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c0628-1879-47ee-98a2-a1b7a8ee76db None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:46.702904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c0628-1879-47ee-98a2-a1b7a8ee76db None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:46.703686 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c0628-1879-47ee-98a2-a1b7a8ee76db None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:46.703988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c0628-1879-47ee-98a2-a1b7a8ee76db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:46.706556 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c0628-1879-47ee-98a2-a1b7a8ee76db None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:46.715528 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62461aae-a3ad-4a66-849c-8a5ce731362b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:46.716263 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62461aae-a3ad-4a66-849c-8a5ce731362b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:46.716546 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62461aae-a3ad-4a66-849c-8a5ce731362b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:46.716820 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62461aae-a3ad-4a66-849c-8a5ce731362b None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:46.717571 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62461aae-a3ad-4a66-849c-8a5ce731362b None admin] RBAC: Adding query filter params (name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:46.717881 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62461aae-a3ad-4a66-849c-8a5ce731362b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:46.719252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62461aae-a3ad-4a66-849c-8a5ce731362b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:46.729538 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 314/644] 10.222.0.47 () {56 vars in 1137 bytes} [Thu Aug 4 04:26:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:46.729538 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c0628-1879-47ee-98a2-a1b7a8ee76db None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:46.729811 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 331/645] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Aug 4 04:26:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:46.730079 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62461aae-a3ad-4a66-849c-8a5ce731362b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:46.798278 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:46.839917 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62461aae-a3ad-4a66-849c-8a5ce731362b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:46.847822 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c0628-1879-47ee-98a2-a1b7a8ee76db None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:46.867322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:46.879370 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37917a-5433-4ad3-a9be-c1a69622f2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:46.880371 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37917a-5433-4ad3-a9be-c1a69622f2bd None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:46.880927 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37917a-5433-4ad3-a9be-c1a69622f2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:46.881211 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37917a-5433-4ad3-a9be-c1a69622f2bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:46.892494 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef75831-4fbb-4c29-8b4c-64c7e18a233a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:46.893147 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef75831-4fbb-4c29-8b4c-64c7e18a233a None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:46.893300 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d37917a-5433-4ad3-a9be-c1a69622f2bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:46.893470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef75831-4fbb-4c29-8b4c-64c7e18a233a None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:46.893725 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef75831-4fbb-4c29-8b4c-64c7e18a233a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:46.900663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-2d37917a-5433-4ad3-a9be-c1a69622f2bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:46.900663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:46.900663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:46.900663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:46.900663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:46.900663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:46.900663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:46.900663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:46.900663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:46.900663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:46.900663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:46.900663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:46.900663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:46.900663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:46.900663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:46.900663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:46.901631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:46.901631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:46.901631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:46.901631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:46.901631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:46.901631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:46.901631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:46.901631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:46.901631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:46.901631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:46.901631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:46.901631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:46.901631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:46.901631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:46.901631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:46.901631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:46.907621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:46.907621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:46.907621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:46.907621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:46.907621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:46.907621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:46.907621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:46.907621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:46.907621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:46.907621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:46.907621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:46.907621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:46.908350 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef75831-4fbb-4c29-8b4c-64c7e18a233a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:46.908350 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef75831-4fbb-4c29-8b4c-64c7e18a233a None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:46.908496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 332/646] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Aug 4 04:26:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:46.908496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37917a-5433-4ad3-a9be-c1a69622f2bd None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:46.930063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-eef75831-4fbb-4c29-8b4c-64c7e18a233a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:46.930063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:46.930063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:46.930063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:46.930063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:46.930063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:46.930063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:46.930063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:46.930063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:46.930063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:46.930063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:46.930063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:46.930063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:26:46.930063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:26:46.930063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:26:46.931029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:26:46.931029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:46.931029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:46.931029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:46.931029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:46.931029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:46.931029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:46.931029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:46.931029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:46.931029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:46.931029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:46.931029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:46.931029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:46.931029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:46.931029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:46.931029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:46.931029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:46.931915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:26:46.931915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:26:46.931915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:26:46.931915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:26:46.931915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:26:46.931915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:26:46.931915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:46.931915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:46.931915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 315/647] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:46.932437 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef75831-4fbb-4c29-8b4c-64c7e18a233a None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:47.032150 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37917a-5433-4ad3-a9be-c1a69622f2bd None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:47.042721 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef75831-4fbb-4c29-8b4c-64c7e18a233a None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:47.062219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:47.077335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcce325-e66a-458e-bc9e-f20d3dd59245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:47.078300 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcce325-e66a-458e-bc9e-f20d3dd59245 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:47.078748 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcce325-e66a-458e-bc9e-f20d3dd59245 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:47.079183 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcce325-e66a-458e-bc9e-f20d3dd59245 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:47.082935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa829b-4fca-4f3d-a970-083ee748085f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:47.083678 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa829b-4fca-4f3d-a970-083ee748085f None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:47.083889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa829b-4fca-4f3d-a970-083ee748085f None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:47.084207 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa829b-4fca-4f3d-a970-083ee748085f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:47.095345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fa829b-4fca-4f3d-a970-083ee748085f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:47.097734 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fa829b-4fca-4f3d-a970-083ee748085f None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:47.113961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-e8fa829b-4fca-4f3d-a970-083ee748085f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:47.113961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:47.113961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:47.113961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:47.113961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:47.113961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:47.113961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:47.113961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:47.113961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:47.113961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:47.113961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:47.113961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:47.113961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:47.113961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:47.113961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:47.114969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:47.114969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:47.114969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:47.114969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:47.114969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:47.114969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:47.114969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:47.114969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:47.114969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:47.114969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:47.114969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:47.114969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:47.114969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:47.114969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:47.114969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:47.114969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:47.114969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:47.115868 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:47.115868 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:47.115868 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:47.115868 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:47.115868 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 316/648] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:47.116156 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa829b-4fca-4f3d-a970-083ee748085f None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:47.160911 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:47.179010 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcce325-e66a-458e-bc9e-f20d3dd59245 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:47.180476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcce325-e66a-458e-bc9e-f20d3dd59245 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:47.247737 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa829b-4fca-4f3d-a970-083ee748085f None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:47.289218 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25063d5e-af19-4802-8f8a-1bae8a87b392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:47.289999 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25063d5e-af19-4802-8f8a-1bae8a87b392 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:47.290283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25063d5e-af19-4802-8f8a-1bae8a87b392 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:47.290558 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25063d5e-af19-4802-8f8a-1bae8a87b392 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:47.291286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25063d5e-af19-4802-8f8a-1bae8a87b392 None admin] RBAC: Adding query filter params (name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:47.291590 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25063d5e-af19-4802-8f8a-1bae8a87b392 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:47.292517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 333/649] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:26:47.292951 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25063d5e-af19-4802-8f8a-1bae8a87b392 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:47.298318 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcce325-e66a-458e-bc9e-f20d3dd59245 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:47.302502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 317/650] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Aug 4 04:26:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:47.303349 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25063d5e-af19-4802-8f8a-1bae8a87b392 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:47.342274 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:47.399785 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:47.422837 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcce325-e66a-458e-bc9e-f20d3dd59245 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:47.439147 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25063d5e-af19-4802-8f8a-1bae8a87b392 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:47.467696 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ac555-7e91-495f-8a72-a9a75882130e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:47.468522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ac555-7e91-495f-8a72-a9a75882130e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:47.469043 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ac555-7e91-495f-8a72-a9a75882130e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:47.469473 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ac555-7e91-495f-8a72-a9a75882130e None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:47.470404 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ac555-7e91-495f-8a72-a9a75882130e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:47.471012 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ac555-7e91-495f-8a72-a9a75882130e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:47.473533 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ac555-7e91-495f-8a72-a9a75882130e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:47.482313 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d991cc8-e802-402f-b74f-7097f332b02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:47.482961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d991cc8-e802-402f-b74f-7097f332b02f None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:47.483274 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d991cc8-e802-402f-b74f-7097f332b02f None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:47.484210 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d991cc8-e802-402f-b74f-7097f332b02f None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:47.484958 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d991cc8-e802-402f-b74f-7097f332b02f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:47.485357 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d991cc8-e802-402f-b74f-7097f332b02f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:47.488195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d991cc8-e802-402f-b74f-7097f332b02f None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:47.497290 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 334/651] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:26:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:47.497290 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ac555-7e91-495f-8a72-a9a75882130e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:47.506578 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 318/652] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:47.507168 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d991cc8-e802-402f-b74f-7097f332b02f None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:47.618146 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:47.620143 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ac555-7e91-495f-8a72-a9a75882130e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:47.628182 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d991cc8-e802-402f-b74f-7097f332b02f None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:47.657686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42635bbc-7d71-4c78-9c79-f715df61198a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:47.658974 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42635bbc-7d71-4c78-9c79-f715df61198a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:47.659261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42635bbc-7d71-4c78-9c79-f715df61198a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:47.659543 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42635bbc-7d71-4c78-9c79-f715df61198a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:47.667665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a004a-54ad-4151-afe2-df249a473fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:47.668227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a004a-54ad-4151-afe2-df249a473fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:47.668512 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a004a-54ad-4151-afe2-df249a473fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:47.668795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a004a-54ad-4151-afe2-df249a473fc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:47.671427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42635bbc-7d71-4c78-9c79-f715df61198a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:47.673606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42635bbc-7d71-4c78-9c79-f715df61198a None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:47.680829 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a004a-54ad-4151-afe2-df249a473fc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:47.682326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a004a-54ad-4151-afe2-df249a473fc6 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:47.685781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:47.696279 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-42635bbc-7d71-4c78-9c79-f715df61198a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:47.696279 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:47.696279 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:47.696279 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:47.696279 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:47.696279 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:47.696279 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:47.696279 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:47.696279 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:47.696279 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:47.696279 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:47.696279 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:47.696279 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:47.696279 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:47.696279 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:47.697282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:47.697282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:47.697282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:47.697282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:47.697282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:47.697282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:47.697282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:47.697282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:47.697282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:47.697282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:47.697282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:47.697282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:47.697282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:47.697282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:47.697282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:47.697282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:47.697282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:47.698193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:47.698193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:47.698193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:47.698193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:47.698193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 335/653] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:47.698597 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-663a004a-54ad-4151-afe2-df249a473fc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:47.698597 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:47.698597 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:47.698597 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:47.698597 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:47.698597 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:47.698597 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:47.698597 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:47.698597 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:47.698597 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:47.698597 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:47.698597 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:47.698597 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:47.698597 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:47.698597 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:47.699398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:47.699398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:47.699398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:47.699398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:47.699398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:47.699398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:47.699398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:47.699398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:47.699398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:47.699398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:47.699398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:47.699398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:47.699398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:47.699398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:47.699398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:47.699398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:47.699398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:47.706567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:47.706567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:47.706567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:47.706567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:47.707126 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42635bbc-7d71-4c78-9c79-f715df61198a None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:47.707292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 319/654] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:47.707292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a004a-54ad-4151-afe2-df249a473fc6 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:47.833796 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a004a-54ad-4151-afe2-df249a473fc6 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:47.838759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42635bbc-7d71-4c78-9c79-f715df61198a None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:47.870821 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60078cf-b2a9-4835-b9da-4104cf3c5e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:47.871736 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60078cf-b2a9-4835-b9da-4104cf3c5e2a None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:47.872167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60078cf-b2a9-4835-b9da-4104cf3c5e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:47.872441 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60078cf-b2a9-4835-b9da-4104cf3c5e2a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:47.879409 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08cece1-812a-4712-af24-640862b9f6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:47.880026 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08cece1-812a-4712-af24-640862b9f6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:47.880311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08cece1-812a-4712-af24-640862b9f6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:47.880582 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08cece1-812a-4712-af24-640862b9f6b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:47.881377 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08cece1-812a-4712-af24-640862b9f6b5 None admin] RBAC: Adding query filter params (role.id=cc86b64693b6417e90ad9361460574b0, scope.project.id=f37a5f3140ec431da8ea14568654114a, user.id=caa2901451aa4554bdfcac1bddbc620a) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:47.881731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08cece1-812a-4712-af24-640862b9f6b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:47.882023 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60078cf-b2a9-4835-b9da-4104cf3c5e2a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:47.883173 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08cece1-812a-4712-af24-640862b9f6b5 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:47.883425 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60078cf-b2a9-4835-b9da-4104cf3c5e2a None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:47.889212 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 336/655] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:26:47 2022] GET /identity/v3/role_assignments?user.id=caa2901451aa4554bdfcac1bddbc620a&scope.project.id=f37a5f3140ec431da8ea14568654114a&role.id=cc86b64693b6417e90ad9361460574b0 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:47.894518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08cece1-812a-4712-af24-640862b9f6b5 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:47.896438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-a60078cf-b2a9-4835-b9da-4104cf3c5e2a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 04:26:47.896438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:47.896438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:47.896438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:47.896438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:47.896438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:47.896438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:47.896438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:47.896438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:47.896438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:47.896438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:47.896438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:47.896438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:26:47.896438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:26:47.896438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:26:47.896438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:26:47.897352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:47.897352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:47.897352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:26:47.897352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:47.897352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:26:47.897352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:47.897352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:47.897352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:47.897352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:47.897352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:47.897352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:47.897352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:47.897352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:47.897352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:47.897352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:47.897352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:47.897352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:47.897352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:47.898319 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:47.898319 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:47.898319 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:26:47.898319 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:26:47.898319 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:26:47.898319 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:26:47.898319 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:26:47.898319 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:26:47.898319 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 04:26:47.898319 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:47.902825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 320/656] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:47.903817 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60078cf-b2a9-4835-b9da-4104cf3c5e2a None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:47.909212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:47.997012 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:48.036495 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60078cf-b2a9-4835-b9da-4104cf3c5e2a None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:48.056431 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08cece1-812a-4712-af24-640862b9f6b5 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:48.085377 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805098d-fa19-4d10-818e-f645eac06f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:48.086336 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805098d-fa19-4d10-818e-f645eac06f40 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:48.086336 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805098d-fa19-4d10-818e-f645eac06f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:48.086531 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805098d-fa19-4d10-818e-f645eac06f40 None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:48.087139 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805098d-fa19-4d10-818e-f645eac06f40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:48.087448 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805098d-fa19-4d10-818e-f645eac06f40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:48.088824 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805098d-fa19-4d10-818e-f645eac06f40 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:48.102734 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 321/657] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:48.103600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805098d-fa19-4d10-818e-f645eac06f40 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:48.108714 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757a718-3d23-4052-830f-acb1e4fda001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:48.109656 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757a718-3d23-4052-830f-acb1e4fda001 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:48.109941 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757a718-3d23-4052-830f-acb1e4fda001 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:48.110213 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757a718-3d23-4052-830f-acb1e4fda001 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:48.110920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d757a718-3d23-4052-830f-acb1e4fda001 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:48.111936 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d757a718-3d23-4052-830f-acb1e4fda001 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:48.113907 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d757a718-3d23-4052-830f-acb1e4fda001 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:48.133341 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 337/658] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:48.134797 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0d0d4-783c-4aac-b4b8-8ca849fb1aad None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:48.135147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0d0d4-783c-4aac-b4b8-8ca849fb1aad None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:48.135363 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0d0d4-783c-4aac-b4b8-8ca849fb1aad None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:48.136661 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 338/659] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:48.137354 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c0d0d4-783c-4aac-b4b8-8ca849fb1aad None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:48.180346 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:48.224523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805098d-fa19-4d10-818e-f645eac06f40 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:48.245450 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c0d0d4-783c-4aac-b4b8-8ca849fb1aad None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:48.266753 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee2153-3b5d-4b20-a4d7-004a4ddea746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:48.267647 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee2153-3b5d-4b20-a4d7-004a4ddea746 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:48.268300 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee2153-3b5d-4b20-a4d7-004a4ddea746 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:48.269254 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee2153-3b5d-4b20-a4d7-004a4ddea746 None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:48.271116 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee2153-3b5d-4b20-a4d7-004a4ddea746 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:48.271821 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:48.272393 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee2153-3b5d-4b20-a4d7-004a4ddea746 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:48.275426 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee2153-3b5d-4b20-a4d7-004a4ddea746 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:48.284600 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c6558-f37c-4a31-a95a-f73f27b01a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:48.285451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c6558-f37c-4a31-a95a-f73f27b01a67 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:48.285853 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c6558-f37c-4a31-a95a-f73f27b01a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:48.286091 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c6558-f37c-4a31-a95a-f73f27b01a67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:48.292935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c6558-f37c-4a31-a95a-f73f27b01a67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:48.294420 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c6558-f37c-4a31-a95a-f73f27b01a67 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:48.296814 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 322/660] 10.222.0.47 () {56 vars in 1137 bytes} [Thu Aug 4 04:26:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:48.297907 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee2153-3b5d-4b20-a4d7-004a4ddea746 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:48.311715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-397c6558-f37c-4a31-a95a-f73f27b01a67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:48.311715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:48.311715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:48.311715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:48.311715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:48.311715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:48.311715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:48.311715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:48.311715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:48.311715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:48.311715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:48.311715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:48.311715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:26:48.311715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:26:48.311715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:26:48.312806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:26:48.312806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:48.312806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:48.312806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:48.312806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:48.312806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:48.312806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:48.312806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:48.312806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:48.312806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:48.312806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:48.312806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:48.312806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:48.312806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:48.312806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:48.312806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:48.312806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:48.313887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:26:48.313887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:26:48.313887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:26:48.313887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:26:48.313887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:26:48.313887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:26:48.313887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:48.313887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:48.317993 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 339/661] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:48.319192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1136887-dbba-476f-8b8f-e7780f0c3530 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:48.319483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1136887-dbba-476f-8b8f-e7780f0c3530 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:48.319757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1136887-dbba-476f-8b8f-e7780f0c3530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:48.337687 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-e1136887-dbba-476f-8b8f-e7780f0c3530 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:48.338967 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e1136887-dbba-476f-8b8f-e7780f0c3530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:48.410221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 340/662] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:48.411399 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1136887-dbba-476f-8b8f-e7780f0c3530 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:48.423243 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee2153-3b5d-4b20-a4d7-004a4ddea746 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:48.450056 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:48.463429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12067f-687b-464e-a1ed-2de8fe310bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:48.464591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12067f-687b-464e-a1ed-2de8fe310bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:48.464975 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12067f-687b-464e-a1ed-2de8fe310bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:48.469274 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12067f-687b-464e-a1ed-2de8fe310bc3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:48.476635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12067f-687b-464e-a1ed-2de8fe310bc3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:48.478119 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12067f-687b-464e-a1ed-2de8fe310bc3 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:48.490861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-ef12067f-687b-464e-a1ed-2de8fe310bc3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 04:26:48.490861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:48.490861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:48.490861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:48.490861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:48.490861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:48.490861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:48.490861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:48.490861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:48.490861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:48.490861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:48.490861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:48.490861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:26:48.490861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:26:48.490861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:26:48.490861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:26:48.492004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:48.492004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:48.492004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:26:48.492004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:48.492004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:26:48.492004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:48.492004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:48.492004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:48.492004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:48.492004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:48.492004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:48.492004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:48.492004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:48.492004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:48.492004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:48.492004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:48.492004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:48.492004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:48.493257 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:48.493257 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:48.493257 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:26:48.493257 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:26:48.493257 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:26:48.493257 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:26:48.493257 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:26:48.493257 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:26:48.493257 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 04:26:48.493257 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:48.496661 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 323/663] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Aug 4 04:26:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:48.497492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12067f-687b-464e-a1ed-2de8fe310bc3 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:48.552825 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1136887-dbba-476f-8b8f-e7780f0c3530 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:48.568630 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:48.591102 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84ae49-8436-48d9-add6-e6d440ac462f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:48.591803 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84ae49-8436-48d9-add6-e6d440ac462f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:48.592150 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84ae49-8436-48d9-add6-e6d440ac462f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:48.592489 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84ae49-8436-48d9-add6-e6d440ac462f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:48.615473 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12067f-687b-464e-a1ed-2de8fe310bc3 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:48.652654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1db62-a93c-4513-aba1-1df487610807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:48.653310 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1db62-a93c-4513-aba1-1df487610807 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:48.653528 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1db62-a93c-4513-aba1-1df487610807 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:48.653921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1db62-a93c-4513-aba1-1df487610807 None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:48.654573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c1db62-a93c-4513-aba1-1df487610807 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:48.654871 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c1db62-a93c-4513-aba1-1df487610807 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:48.656275 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c1db62-a93c-4513-aba1-1df487610807 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:48.663935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84ae49-8436-48d9-add6-e6d440ac462f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:48.665970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84ae49-8436-48d9-add6-e6d440ac462f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:48.668414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 324/664] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:26:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:48.669144 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1db62-a93c-4513-aba1-1df487610807 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:48.715933 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:48.777435 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 341/665] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:26:48.778541 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84ae49-8436-48d9-add6-e6d440ac462f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:48.778843 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1db62-a93c-4513-aba1-1df487610807 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:48.793797 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:48.820780 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa691a5-6c07-48e8-b53e-4d005668c387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:48.821691 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa691a5-6c07-48e8-b53e-4d005668c387 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:48.821974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa691a5-6c07-48e8-b53e-4d005668c387 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:48.822249 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa691a5-6c07-48e8-b53e-4d005668c387 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:48.833612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa691a5-6c07-48e8-b53e-4d005668c387 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:48.839007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-caa691a5-6c07-48e8-b53e-4d005668c387 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:48.839007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:48.839007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:48.839007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:48.839007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:48.839007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:48.839007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:48.839007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:48.839007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:48.839007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:48.839007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:48.839007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:48.839007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:48.839007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:48.839007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:48.839007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:48.840086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:48.840086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:48.840086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:48.840086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:48.840086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:48.840086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:48.840086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:48.840086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:48.840086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:48.840086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:48.840086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:48.840086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:48.840086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:48.840086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:48.840086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:48.840086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:48.841130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:48.841130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:48.841130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:48.841130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:48.841130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:48.841130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:48.841130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:48.841130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:48.841130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:48.841130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:48.841130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:48.841130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:48.841130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 325/666] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Aug 4 04:26:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:48.841974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa691a5-6c07-48e8-b53e-4d005668c387 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:48.887218 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84ae49-8436-48d9-add6-e6d440ac462f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:48.930261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f33597-cb0c-452d-9002-c2822b1c8acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:48.931142 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f33597-cb0c-452d-9002-c2822b1c8acb None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:48.931423 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f33597-cb0c-452d-9002-c2822b1c8acb None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:48.931746 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f33597-cb0c-452d-9002-c2822b1c8acb None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:48.932544 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f33597-cb0c-452d-9002-c2822b1c8acb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:48.932851 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f33597-cb0c-452d-9002-c2822b1c8acb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:48.934300 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f33597-cb0c-452d-9002-c2822b1c8acb None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:48.953709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 342/667] 10.222.0.47 () {56 vars in 1139 bytes} [Thu Aug 4 04:26:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:48.954241 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa691a5-6c07-48e8-b53e-4d005668c387 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:48.955016 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f33597-cb0c-452d-9002-c2822b1c8acb None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:48.956592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:48.997386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5472f6-f742-477e-9ac6-526224905ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:48.998034 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5472f6-f742-477e-9ac6-526224905ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:48.998316 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5472f6-f742-477e-9ac6-526224905ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:48.998585 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5472f6-f742-477e-9ac6-526224905ed2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:48.999375 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5472f6-f742-477e-9ac6-526224905ed2 None admin] RBAC: Adding query filter params (role.id=cc86b64693b6417e90ad9361460574b0, scope.project.id=f37a5f3140ec431da8ea14568654114a, user.id=b77a955866c54a43a7aaa271fefdec57) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:48.999686 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5472f6-f742-477e-9ac6-526224905ed2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:49.001093 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5472f6-f742-477e-9ac6-526224905ed2 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:49.007785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 326/668] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:26:48 2022] GET /identity/v3/role_assignments?user.id=b77a955866c54a43a7aaa271fefdec57&scope.project.id=f37a5f3140ec431da8ea14568654114a&role.id=cc86b64693b6417e90ad9361460574b0 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:49.008820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5472f6-f742-477e-9ac6-526224905ed2 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:49.055930 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:49.096274 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f33597-cb0c-452d-9002-c2822b1c8acb None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:49.151054 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5472f6-f742-477e-9ac6-526224905ed2 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:49.159336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe657b-4f27-4bf4-a226-90702a10bd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:49.160883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe657b-4f27-4bf4-a226-90702a10bd2d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:49.161261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe657b-4f27-4bf4-a226-90702a10bd2d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:49.161675 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe657b-4f27-4bf4-a226-90702a10bd2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:49.178351 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe657b-4f27-4bf4-a226-90702a10bd2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:49.179819 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe657b-4f27-4bf4-a226-90702a10bd2d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:49.198502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-9dfe657b-4f27-4bf4-a226-90702a10bd2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:49.198502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:49.198502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:49.198502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:49.198502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:49.198502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:49.198502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:49.198502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:49.198502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:49.198502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:49.198502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:49.198502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:49.198502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:49.198502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:49.198502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:49.199606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:49.199606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:49.199606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:49.199606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:49.199606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:49.199606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:49.199606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:49.199606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:49.199606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:49.199606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:49.199606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:49.199606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:49.199606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:49.199606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:49.199606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:49.199606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:49.199606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:49.200756 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:49.200756 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:49.200756 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:49.200756 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:49.202620 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 343/669] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:49.203564 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe657b-4f27-4bf4-a226-90702a10bd2d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:49.204267 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fb0a0-b475-4890-ad09-314ef3dd353d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:49.205248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fb0a0-b475-4890-ad09-314ef3dd353d None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:49.205745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fb0a0-b475-4890-ad09-314ef3dd353d None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:49.206191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fb0a0-b475-4890-ad09-314ef3dd353d None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:49.207418 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fb0a0-b475-4890-ad09-314ef3dd353d None admin] RBAC: Adding query filter params (name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:49.207852 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fb0a0-b475-4890-ad09-314ef3dd353d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:49.209305 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fb0a0-b475-4890-ad09-314ef3dd353d None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:49.219338 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 327/670] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Aug 4 04:26:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:49.222169 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fb0a0-b475-4890-ad09-314ef3dd353d None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:49.255485 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:49.333174 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe657b-4f27-4bf4-a226-90702a10bd2d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:49.352318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fb0a0-b475-4890-ad09-314ef3dd353d None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:49.360167 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:49.374258 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a88d59-32a9-420c-a244-d403e7ab6b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:49.375061 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a88d59-32a9-420c-a244-d403e7ab6b49 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:49.375347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a88d59-32a9-420c-a244-d403e7ab6b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:49.375622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a88d59-32a9-420c-a244-d403e7ab6b49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:49.405457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b6ad3-2d72-4fc3-ab7f-70d241025533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:49.406680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b6ad3-2d72-4fc3-ab7f-70d241025533 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:49.407155 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b6ad3-2d72-4fc3-ab7f-70d241025533 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:49.407853 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b6ad3-2d72-4fc3-ab7f-70d241025533 None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:49.409315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b6ad3-2d72-4fc3-ab7f-70d241025533 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:49.409797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b6ad3-2d72-4fc3-ab7f-70d241025533 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:49.411828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b6ad3-2d72-4fc3-ab7f-70d241025533 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:49.433681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 328/671] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:49.434542 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b6ad3-2d72-4fc3-ab7f-70d241025533 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:49.466671 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a88d59-32a9-420c-a244-d403e7ab6b49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:49.468806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a88d59-32a9-420c-a244-d403e7ab6b49 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:49.535661 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:49.550562 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b6ad3-2d72-4fc3-ab7f-70d241025533 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:49.588572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f0066-523c-4058-bcec-6bf365dd8a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:49.589455 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f0066-523c-4058-bcec-6bf365dd8a6e None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:49.589903 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f0066-523c-4058-bcec-6bf365dd8a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:49.590350 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f0066-523c-4058-bcec-6bf365dd8a6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:49.601469 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 344/672] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:26:49.602426 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a88d59-32a9-420c-a244-d403e7ab6b49 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:49.605360 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f0066-523c-4058-bcec-6bf365dd8a6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:49.607113 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f0066-523c-4058-bcec-6bf365dd8a6e None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:49.623604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-d88f0066-523c-4058-bcec-6bf365dd8a6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:49.623604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:49.623604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:49.623604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:49.623604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:49.623604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:49.623604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:49.623604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:49.623604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:49.623604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:49.623604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:49.623604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:49.623604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:49.623604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:49.623604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:49.624720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:49.624720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:49.624720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:49.624720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:49.624720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:49.624720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:49.624720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:49.624720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:49.624720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:49.624720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:49.624720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:49.624720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:49.624720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:49.624720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:49.624720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:49.624720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:49.624720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:49.625829 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:49.625829 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:49.625829 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:49.625829 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:49.628934 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 329/673] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:49.630051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f0066-523c-4058-bcec-6bf365dd8a6e None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:49.635906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:49.728471 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a88d59-32a9-420c-a244-d403e7ab6b49 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:49.769451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7be5e-0e39-428c-b743-ca5d693b49cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:49.770162 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7be5e-0e39-428c-b743-ca5d693b49cc None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:49.770445 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7be5e-0e39-428c-b743-ca5d693b49cc None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:49.770767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7be5e-0e39-428c-b743-ca5d693b49cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:49.771167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f0066-523c-4058-bcec-6bf365dd8a6e None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:49.778498 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7be5e-0e39-428c-b743-ca5d693b49cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:49.780189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7be5e-0e39-428c-b743-ca5d693b49cc None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:49.793247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-02b7be5e-0e39-428c-b743-ca5d693b49cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:49.793247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:49.793247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:49.793247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:49.793247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:49.793247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:49.793247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:49.793247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:49.793247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:49.793247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:49.793247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:49.793247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:49.793247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:26:49.793247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:26:49.793247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:26:49.794375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:26:49.794375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:49.794375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:49.794375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:49.794375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:49.794375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:49.794375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:49.794375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:49.794375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:49.794375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:49.794375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:49.794375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:49.794375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:49.794375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:49.794375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:49.794375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:49.794375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:49.795700 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:26:49.795700 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:26:49.795700 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:26:49.795700 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:26:49.795700 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:26:49.795700 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:26:49.795700 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:49.795700 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:49.797521 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 345/674] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:49.798262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7be5e-0e39-428c-b743-ca5d693b49cc None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:49.815931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf85f56-2ba3-406c-bf55-284a9486dfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:49.817043 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf85f56-2ba3-406c-bf55-284a9486dfe6 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:49.818014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf85f56-2ba3-406c-bf55-284a9486dfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:49.818436 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf85f56-2ba3-406c-bf55-284a9486dfe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:49.831135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:49.833501 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf85f56-2ba3-406c-bf55-284a9486dfe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:49.835720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf85f56-2ba3-406c-bf55-284a9486dfe6 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:49.855086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-4bf85f56-2ba3-406c-bf55-284a9486dfe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:49.855086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:49.855086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:49.855086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:49.855086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:49.855086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:49.855086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:49.855086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:49.855086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:49.855086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:49.855086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:49.855086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:49.855086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:49.855086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:49.855086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:49.856167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:49.856167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:49.856167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:49.856167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:49.856167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:49.856167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:49.856167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:49.856167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:49.856167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:49.856167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:49.856167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:49.856167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:49.856167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:49.856167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:49.856167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:49.856167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:49.856167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:49.857256 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:49.857256 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:49.857256 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:49.857256 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:49.860581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 330/675] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:49.862796 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf85f56-2ba3-406c-bf55-284a9486dfe6 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:49.904719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:49.933589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7be5e-0e39-428c-b743-ca5d693b49cc None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:49.975340 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0237ca-0355-4d3e-b829-d4343c6cdf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:49.976126 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0237ca-0355-4d3e-b829-d4343c6cdf4f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:49.976499 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0237ca-0355-4d3e-b829-d4343c6cdf4f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:49.976887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0237ca-0355-4d3e-b829-d4343c6cdf4f None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:49.977711 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0237ca-0355-4d3e-b829-d4343c6cdf4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:49.978112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0237ca-0355-4d3e-b829-d4343c6cdf4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:49.979621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0237ca-0355-4d3e-b829-d4343c6cdf4f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:49.994108 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 346/676] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:49.995065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0237ca-0355-4d3e-b829-d4343c6cdf4f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:50.019422 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf85f56-2ba3-406c-bf55-284a9486dfe6 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:50.062996 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbb6c7-82ea-4346-9959-81f7e524a200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:50.063611 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbb6c7-82ea-4346-9959-81f7e524a200 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:50.063888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbb6c7-82ea-4346-9959-81f7e524a200 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:50.064822 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbb6c7-82ea-4346-9959-81f7e524a200 None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:50.065648 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbb6c7-82ea-4346-9959-81f7e524a200 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:50.065954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbb6c7-82ea-4346-9959-81f7e524a200 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:50.067363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbb6c7-82ea-4346-9959-81f7e524a200 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:50.079409 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 331/677] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:26:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:50.080978 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbb6c7-82ea-4346-9959-81f7e524a200 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:50.119976 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0237ca-0355-4d3e-b829-d4343c6cdf4f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:50.138754 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:50.159232 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13014c59-f795-4c93-8d89-8896a1b00732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:50.159847 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13014c59-f795-4c93-8d89-8896a1b00732 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:50.160124 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13014c59-f795-4c93-8d89-8896a1b00732 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:50.160396 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13014c59-f795-4c93-8d89-8896a1b00732 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:50.161082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13014c59-f795-4c93-8d89-8896a1b00732 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:50.161386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13014c59-f795-4c93-8d89-8896a1b00732 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:50.162768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13014c59-f795-4c93-8d89-8896a1b00732 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:50.174964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 347/678] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:50.176244 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64281ca-9b88-4c53-86db-d438414235ef None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:50.176531 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64281ca-9b88-4c53-86db-d438414235ef None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:50.176801 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64281ca-9b88-4c53-86db-d438414235ef None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:50.178191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 348/679] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:50.178723 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64281ca-9b88-4c53-86db-d438414235ef None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:50.181838 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:50.199554 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbb6c7-82ea-4346-9959-81f7e524a200 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:50.236859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee7537-7ba7-4441-8398-eaf68543e68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:50.237432 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee7537-7ba7-4441-8398-eaf68543e68c None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:50.237812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee7537-7ba7-4441-8398-eaf68543e68c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:50.238012 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee7537-7ba7-4441-8398-eaf68543e68c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:50.245576 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee7537-7ba7-4441-8398-eaf68543e68c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:50.247000 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee7537-7ba7-4441-8398-eaf68543e68c None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:50.259386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-4eee7537-7ba7-4441-8398-eaf68543e68c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 04:26:50.259386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:50.259386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:50.259386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:50.259386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:50.259386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:50.259386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:50.259386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:50.259386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:50.259386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:50.259386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:50.259386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:50.259386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:26:50.259386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:26:50.259386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:26:50.259386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:26:50.260351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:50.260351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:50.260351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:26:50.260351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:50.260351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:26:50.260351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:50.260351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:50.260351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:50.260351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:50.260351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:50.260351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:50.260351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:50.260351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:50.260351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:50.260351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:50.260351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:50.260351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:50.260351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:50.261932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:50.261932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:50.261932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:26:50.261932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:26:50.261932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:26:50.261932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:26:50.261932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:26:50.261932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:26:50.261932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 04:26:50.261932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:50.261932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 332/680] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:26:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:50.261932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee7537-7ba7-4441-8398-eaf68543e68c None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:50.293042 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64281ca-9b88-4c53-86db-d438414235ef None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:50.332898 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b789bc7-d2ff-4355-80dc-85de578945b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:50.333516 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b789bc7-d2ff-4355-80dc-85de578945b4 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:50.333828 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b789bc7-d2ff-4355-80dc-85de578945b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:50.334098 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b789bc7-d2ff-4355-80dc-85de578945b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:50.334900 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b789bc7-d2ff-4355-80dc-85de578945b4 None admin] RBAC: Adding query filter params (role.id=cc86b64693b6417e90ad9361460574b0, scope.project.id=f37a5f3140ec431da8ea14568654114a, user.id=8b1d67b28dca49ca9cb0ff28cad62aa1) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:50.335209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b789bc7-d2ff-4355-80dc-85de578945b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:50.336644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b789bc7-d2ff-4355-80dc-85de578945b4 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:50.343805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 349/681] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:26:50 2022] GET /identity/v3/role_assignments?user.id=8b1d67b28dca49ca9cb0ff28cad62aa1&scope.project.id=f37a5f3140ec431da8ea14568654114a&role.id=cc86b64693b6417e90ad9361460574b0 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:50.345266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb440a26-ed62-42c2-b3e0-bbab6113458d None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:50.346307 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb440a26-ed62-42c2-b3e0-bbab6113458d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:50.346959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb440a26-ed62-42c2-b3e0-bbab6113458d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:50.370116 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-bb440a26-ed62-42c2-b3e0-bbab6113458d None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:50.371704 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-bb440a26-ed62-42c2-b3e0-bbab6113458d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:50.384522 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee7537-7ba7-4441-8398-eaf68543e68c None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:50.404923 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:50.434794 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74019222-fe70-4a14-beaa-01d49ac30b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:50.435460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74019222-fe70-4a14-beaa-01d49ac30b2f None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:50.435738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74019222-fe70-4a14-beaa-01d49ac30b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:50.436011 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74019222-fe70-4a14-beaa-01d49ac30b2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:50.444300 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74019222-fe70-4a14-beaa-01d49ac30b2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:50.445835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74019222-fe70-4a14-beaa-01d49ac30b2f None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:50.459572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-74019222-fe70-4a14-beaa-01d49ac30b2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:50.459572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:50.459572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:50.459572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:50.459572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:50.459572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:50.459572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:50.459572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:50.459572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:50.459572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:50.459572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:50.459572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:50.459572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:26:50.459572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:26:50.459572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:26:50.460615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:26:50.460615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:50.460615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:50.460615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:50.460615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:50.460615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:50.460615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:50.460615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:50.460615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:50.460615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:50.460615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:50.460615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:50.460615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:50.460615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:50.460615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:50.460615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:50.460615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:50.462117 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:26:50.462117 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:26:50.462117 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:26:50.462117 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:26:50.462117 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:26:50.462117 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:26:50.462117 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:50.462117 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:50.462117 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 333/682] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:50.462117 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74019222-fe70-4a14-beaa-01d49ac30b2f None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:50.465276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:50.478130 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 350/683] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:50.479753 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb440a26-ed62-42c2-b3e0-bbab6113458d None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:50.579117 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74019222-fe70-4a14-beaa-01d49ac30b2f None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:50.612731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb440a26-ed62-42c2-b3e0-bbab6113458d None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:50.621793 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799af09c-6b73-4ad1-9bd9-dded303214b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:50.622427 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799af09c-6b73-4ad1-9bd9-dded303214b2 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:50.622710 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799af09c-6b73-4ad1-9bd9-dded303214b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:50.622984 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799af09c-6b73-4ad1-9bd9-dded303214b2 None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:50.624151 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799af09c-6b73-4ad1-9bd9-dded303214b2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:50.624465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799af09c-6b73-4ad1-9bd9-dded303214b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:50.626243 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799af09c-6b73-4ad1-9bd9-dded303214b2 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:50.645912 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 334/684] 10.222.0.47 () {56 vars in 1135 bytes} [Thu Aug 4 04:26:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:50.646434 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799af09c-6b73-4ad1-9bd9-dded303214b2 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:50.659216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328ffaf-7322-42de-b810-81ab24decf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:50.660235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328ffaf-7322-42de-b810-81ab24decf88 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:50.661136 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328ffaf-7322-42de-b810-81ab24decf88 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:50.661650 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328ffaf-7322-42de-b810-81ab24decf88 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:50.662972 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328ffaf-7322-42de-b810-81ab24decf88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:50.663480 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328ffaf-7322-42de-b810-81ab24decf88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:50.665789 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328ffaf-7322-42de-b810-81ab24decf88 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:50.685576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 351/685] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:26:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:50.685576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328ffaf-7322-42de-b810-81ab24decf88 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:50.693665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:50.771402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:50.801894 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799af09c-6b73-4ad1-9bd9-dded303214b2 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:50.819461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328ffaf-7322-42de-b810-81ab24decf88 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:50.845925 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e1b67-1614-428f-aa3e-11127657a193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:50.847069 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e1b67-1614-428f-aa3e-11127657a193 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:50.847558 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e1b67-1614-428f-aa3e-11127657a193 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:50.848082 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e1b67-1614-428f-aa3e-11127657a193 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:50.855919 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e5d83b-8ea2-4b0f-a851-96cfd248627d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:50.856548 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e5d83b-8ea2-4b0f-a851-96cfd248627d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:50.856829 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e5d83b-8ea2-4b0f-a851-96cfd248627d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:50.857104 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e5d83b-8ea2-4b0f-a851-96cfd248627d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:50.862612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e1b67-1614-428f-aa3e-11127657a193 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:50.868037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-869e1b67-1614-428f-aa3e-11127657a193 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:26:50.868037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:50.868037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:50.868037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:50.868037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:50.868037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:50.868037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:50.868037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:50.868037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:50.868037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:50.868037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:50.868037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:50.868037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:50.868037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:50.868037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:50.868037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:50.868985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:50.868985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:50.868985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:50.868985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:50.868985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:50.868985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:50.868985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:50.868985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:50.868985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:50.868985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:50.868985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:50.868985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:50.868985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:50.868985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:50.868985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:50.868985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:50.869870 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:50.869870 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:50.869870 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:50.869870 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:50.869870 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:50.869870 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:50.869870 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:50.869870 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:50.869870 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:50.869870 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:50.869870 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:26:50.869870 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:50.872701 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 335/686] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:26:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:50.873878 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e1b67-1614-428f-aa3e-11127657a193 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:50.940117 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e5d83b-8ea2-4b0f-a851-96cfd248627d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:50.941620 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e5d83b-8ea2-4b0f-a851-96cfd248627d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:50.982286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:50.992846 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e1b67-1614-428f-aa3e-11127657a193 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:51.001464 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:51.030237 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803fba1-1816-4105-bfa1-24ab73478fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:51.031242 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803fba1-1816-4105-bfa1-24ab73478fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:51.031731 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803fba1-1816-4105-bfa1-24ab73478fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:51.032207 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803fba1-1816-4105-bfa1-24ab73478fd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:51.033932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803fba1-1816-4105-bfa1-24ab73478fd2 None admin] RBAC: Adding query filter params (role.id=cc86b64693b6417e90ad9361460574b0, scope.project.id=f37a5f3140ec431da8ea14568654114a, user.id=1a2a647f97fc4d369f562e8b5a4bf68c) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:51.034455 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803fba1-1816-4105-bfa1-24ab73478fd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:51.036154 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803fba1-1816-4105-bfa1-24ab73478fd2 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:51.049708 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 336/688] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:26:50 2022] GET /identity/v3/role_assignments?user.id=1a2a647f97fc4d369f562e8b5a4bf68c&scope.project.id=f37a5f3140ec431da8ea14568654114a&role.id=cc86b64693b6417e90ad9361460574b0 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:51.051417 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803fba1-1816-4105-bfa1-24ab73478fd2 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:51.053875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 352/688] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:26:51.055637 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e5d83b-8ea2-4b0f-a851-96cfd248627d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:51.181914 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e5d83b-8ea2-4b0f-a851-96cfd248627d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:51.223224 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7c319-6d9f-44de-bbbc-86549bc651b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:51.223887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7c319-6d9f-44de-bbbc-86549bc651b2 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:51.224167 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7c319-6d9f-44de-bbbc-86549bc651b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:51.224437 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7c319-6d9f-44de-bbbc-86549bc651b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:51.236756 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d7c319-6d9f-44de-bbbc-86549bc651b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:51.238297 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d7c319-6d9f-44de-bbbc-86549bc651b2 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:51.255639 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-99d7c319-6d9f-44de-bbbc-86549bc651b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:51.255639 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:51.255639 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:51.255639 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:51.255639 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:51.255639 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:51.255639 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:51.255639 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:51.255639 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:51.255639 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:51.255639 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:51.255639 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:51.255639 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:51.255639 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:51.255639 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:51.256595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:51.256595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:51.256595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:51.256595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:51.256595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:51.256595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:51.256595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:51.256595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:51.256595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:51.256595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:51.256595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:51.256595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:51.256595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:51.256595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:51.256595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:51.256595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:51.256595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:51.257975 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:51.257975 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:51.257975 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:51.257975 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:51.257975 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 353/689] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:51.263985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7c319-6d9f-44de-bbbc-86549bc651b2 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:51.293982 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:51.344939 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803fba1-1816-4105-bfa1-24ab73478fd2 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:51.393705 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da452c4-1ab4-4040-bf15-9113c4e361cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:51.394486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da452c4-1ab4-4040-bf15-9113c4e361cc None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:51.395131 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da452c4-1ab4-4040-bf15-9113c4e361cc None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:51.395412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da452c4-1ab4-4040-bf15-9113c4e361cc None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:51.396417 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da452c4-1ab4-4040-bf15-9113c4e361cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:51.396724 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da452c4-1ab4-4040-bf15-9113c4e361cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:51.398608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da452c4-1ab4-4040-bf15-9113c4e361cc None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:51.404084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7c319-6d9f-44de-bbbc-86549bc651b2 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:51.410026 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 337/690] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:51.411085 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37a6a0-53a7-41e1-a78b-121ecad97954 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:51.411426 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37a6a0-53a7-41e1-a78b-121ecad97954 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:51.411712 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37a6a0-53a7-41e1-a78b-121ecad97954 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:51.413890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 338/691] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:51.416887 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37a6a0-53a7-41e1-a78b-121ecad97954 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:51.443139 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:51.453868 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827bf25-7915-4ba5-a7a2-e93218a69dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:51.454527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827bf25-7915-4ba5-a7a2-e93218a69dcc None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:51.455512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827bf25-7915-4ba5-a7a2-e93218a69dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:51.455806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827bf25-7915-4ba5-a7a2-e93218a69dcc None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:51.456744 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827bf25-7915-4ba5-a7a2-e93218a69dcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:51.457056 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827bf25-7915-4ba5-a7a2-e93218a69dcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:51.458550 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827bf25-7915-4ba5-a7a2-e93218a69dcc None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:51.475116 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 354/692] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:51.476432 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064c166-9614-4487-8172-925bc3343746 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:51.476722 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064c166-9614-4487-8172-925bc3343746 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:51.476995 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064c166-9614-4487-8172-925bc3343746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:51.498343 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-3064c166-9614-4487-8172-925bc3343746 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:51.499554 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3064c166-9614-4487-8172-925bc3343746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:51.564736 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37a6a0-53a7-41e1-a78b-121ecad97954 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:51.579297 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:51.586179 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 355/693] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:51.587304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064c166-9614-4487-8172-925bc3343746 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:51.632412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a40f4b-6554-4772-9e2d-bd31723e615c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:51.633374 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a40f4b-6554-4772-9e2d-bd31723e615c None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:51.633728 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a40f4b-6554-4772-9e2d-bd31723e615c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:51.634005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a40f4b-6554-4772-9e2d-bd31723e615c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:51.648690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a40f4b-6554-4772-9e2d-bd31723e615c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:51.650766 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a40f4b-6554-4772-9e2d-bd31723e615c None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:51.668867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-78a40f4b-6554-4772-9e2d-bd31723e615c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:51.668867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:51.668867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:51.668867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:51.668867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:51.668867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:51.668867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:51.668867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:51.668867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:51.668867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:51.668867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:51.668867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:51.668867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:51.668867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:51.668867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:51.669919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:51.669919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:51.669919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:51.669919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:51.669919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:51.669919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:51.669919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:51.669919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:51.669919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:51.669919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:51.669919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:51.669919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:51.669919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:51.669919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:51.669919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:51.669919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:51.669919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:51.671116 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:51.671116 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:51.671116 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:51.671116 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:51.671116 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 339/694] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:51.671410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a40f4b-6554-4772-9e2d-bd31723e615c None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:51.722431 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064c166-9614-4487-8172-925bc3343746 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:51.746806 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:51.764323 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628af5c-ea12-4830-ad8d-e651ccb11d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:51.765221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628af5c-ea12-4830-ad8d-e651ccb11d05 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:51.765713 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628af5c-ea12-4830-ad8d-e651ccb11d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:51.766133 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628af5c-ea12-4830-ad8d-e651ccb11d05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:51.773811 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b628af5c-ea12-4830-ad8d-e651ccb11d05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:51.776073 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b628af5c-ea12-4830-ad8d-e651ccb11d05 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:51.786108 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a40f4b-6554-4772-9e2d-bd31723e615c None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:51.788715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-b628af5c-ea12-4830-ad8d-e651ccb11d05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:51.788715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:51.788715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:51.788715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:51.788715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:51.788715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:51.788715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:51.788715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:51.788715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:51.788715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:51.788715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:51.788715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:51.788715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:26:51.788715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:26:51.788715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:26:51.789769 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:26:51.789769 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:51.789769 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:51.789769 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:51.789769 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:51.789769 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:51.789769 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:51.789769 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:51.789769 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:51.789769 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:51.789769 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:51.789769 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:51.789769 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:51.789769 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:51.789769 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:51.789769 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:51.789769 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:51.790764 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:26:51.790764 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:26:51.790764 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:26:51.790764 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:26:51.790764 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:26:51.790764 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:26:51.790764 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:51.790764 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:51.792836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 356/695] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:51.793913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628af5c-ea12-4830-ad8d-e651ccb11d05 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:51.827789 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd149415-51ec-4c96-92d4-3095884bb34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:51.828340 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd149415-51ec-4c96-92d4-3095884bb34d None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:51.828690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd149415-51ec-4c96-92d4-3095884bb34d None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:51.828972 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd149415-51ec-4c96-92d4-3095884bb34d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:51.840414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd149415-51ec-4c96-92d4-3095884bb34d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:51.841959 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd149415-51ec-4c96-92d4-3095884bb34d None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:51.858047 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-fd149415-51ec-4c96-92d4-3095884bb34d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:51.858047 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:51.858047 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:51.858047 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:51.858047 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:51.858047 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:51.858047 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:51.858047 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:51.858047 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:51.858047 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:51.858047 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:51.858047 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:51.858047 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:51.858047 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:51.858047 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:51.859185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:51.859185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:51.859185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:51.859185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:51.859185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:51.859185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:51.859185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:51.859185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:51.859185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:51.859185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:51.859185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:51.859185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:51.859185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:51.859185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:51.859185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:51.859185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:51.859185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:51.860141 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:51.860141 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:51.860141 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:51.860141 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:51.864634 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 340/696] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:51.865502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd149415-51ec-4c96-92d4-3095884bb34d None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:51.867043 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:51.913014 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628af5c-ea12-4830-ad8d-e651ccb11d05 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:51.950550 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce41d7-fe26-4dd8-ad14-e9699d4537c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:51.951451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce41d7-fe26-4dd8-ad14-e9699d4537c6 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:51.951884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce41d7-fe26-4dd8-ad14-e9699d4537c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:51.952306 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce41d7-fe26-4dd8-ad14-e9699d4537c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:51.953146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ce41d7-fe26-4dd8-ad14-e9699d4537c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:51.953625 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ce41d7-fe26-4dd8-ad14-e9699d4537c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:51.956076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ce41d7-fe26-4dd8-ad14-e9699d4537c6 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:51.969164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 357/697] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:51.970619 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce41d7-fe26-4dd8-ad14-e9699d4537c6 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:51.973998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd149415-51ec-4c96-92d4-3095884bb34d None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:51.995922 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:52.012340 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf9678-3fe7-4971-8c98-22abd968eafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:52.012962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf9678-3fe7-4971-8c98-22abd968eafb None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:52.013244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf9678-3fe7-4971-8c98-22abd968eafb None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:52.013518 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf9678-3fe7-4971-8c98-22abd968eafb None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:52.014353 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf9678-3fe7-4971-8c98-22abd968eafb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:52.014662 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf9678-3fe7-4971-8c98-22abd968eafb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:52.016140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf9678-3fe7-4971-8c98-22abd968eafb None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:52.034901 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 341/698] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:26:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:52.035818 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf9678-3fe7-4971-8c98-22abd968eafb None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:52.082193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce41d7-fe26-4dd8-ad14-e9699d4537c6 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:52.101991 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:52.121575 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561055c-2596-4813-ba55-ff60b5f66d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:52.122567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561055c-2596-4813-ba55-ff60b5f66d77 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:52.123101 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561055c-2596-4813-ba55-ff60b5f66d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:52.123588 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561055c-2596-4813-ba55-ff60b5f66d77 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:52.124496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561055c-2596-4813-ba55-ff60b5f66d77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:52.125044 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561055c-2596-4813-ba55-ff60b5f66d77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:52.126670 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561055c-2596-4813-ba55-ff60b5f66d77 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:52.140366 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 358/699] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:52.141520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561055c-2596-4813-ba55-ff60b5f66d77 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:52.150846 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf9678-3fe7-4971-8c98-22abd968eafb None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:52.187138 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab5d274-058a-4acc-a04f-fd35375ad472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:52.187787 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5d274-058a-4acc-a04f-fd35375ad472 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:52.188070 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5d274-058a-4acc-a04f-fd35375ad472 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:52.188343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5d274-058a-4acc-a04f-fd35375ad472 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:52.197087 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab5d274-058a-4acc-a04f-fd35375ad472 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:52.198454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab5d274-058a-4acc-a04f-fd35375ad472 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:52.211416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-1ab5d274-058a-4acc-a04f-fd35375ad472 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 04:26:52.211416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:52.211416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:52.211416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:52.211416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:52.211416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:52.211416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:52.211416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:52.211416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:52.211416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:52.211416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:52.211416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:52.211416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:26:52.211416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:26:52.211416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:26:52.211416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:26:52.212378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:52.212378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:52.212378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:26:52.212378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:52.212378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:26:52.212378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:52.212378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:52.212378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:52.212378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:52.212378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:52.212378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:52.212378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:52.212378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:52.212378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:52.212378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:52.212378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:52.212378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:52.212378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:52.213298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:52.213298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:52.213298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:26:52.213298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:26:52.213298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:26:52.213298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:26:52.213298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:26:52.213298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:26:52.213298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 04:26:52.213298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:52.213298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 342/700] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Aug 4 04:26:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:52.213903 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab5d274-058a-4acc-a04f-fd35375ad472 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:52.259203 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:52.260126 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561055c-2596-4813-ba55-ff60b5f66d77 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:52.303846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0584ca2-25da-4d7d-8a63-2bfa03f9de9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:52.304832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0584ca2-25da-4d7d-8a63-2bfa03f9de9e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:52.305324 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0584ca2-25da-4d7d-8a63-2bfa03f9de9e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:52.305959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0584ca2-25da-4d7d-8a63-2bfa03f9de9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:52.306976 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0584ca2-25da-4d7d-8a63-2bfa03f9de9e None admin] RBAC: Adding query filter params (role.id=cc86b64693b6417e90ad9361460574b0, scope.project.id=f37a5f3140ec431da8ea14568654114a, user.id=b1e539006166434790823b839bd745d5) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:52.307494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0584ca2-25da-4d7d-8a63-2bfa03f9de9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:52.309189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0584ca2-25da-4d7d-8a63-2bfa03f9de9e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:52.319816 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 359/701] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:26:52 2022] GET /identity/v3/role_assignments?user.id=b1e539006166434790823b839bd745d5&scope.project.id=f37a5f3140ec431da8ea14568654114a&role.id=cc86b64693b6417e90ad9361460574b0 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:52.320972 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0584ca2-25da-4d7d-8a63-2bfa03f9de9e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:52.359506 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab5d274-058a-4acc-a04f-fd35375ad472 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:52.437995 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:52.449561 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c797da-701a-443f-9f6b-173e0a2fd87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:52.450204 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c797da-701a-443f-9f6b-173e0a2fd87c None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:52.450490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c797da-701a-443f-9f6b-173e0a2fd87c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:52.450770 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c797da-701a-443f-9f6b-173e0a2fd87c None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:52.451515 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c797da-701a-443f-9f6b-173e0a2fd87c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:52.452887 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c797da-701a-443f-9f6b-173e0a2fd87c None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:52.494266 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 343/702] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Aug 4 04:26:52 2022] POST /identity/v3/users => generated 254 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:26:52.529246 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0584ca2-25da-4d7d-8a63-2bfa03f9de9e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:52.572709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9eb76-9588-4ad5-a02a-68710d1212b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:52.573766 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9eb76-9588-4ad5-a02a-68710d1212b8 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:52.574279 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9eb76-9588-4ad5-a02a-68710d1212b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:52.575028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9eb76-9588-4ad5-a02a-68710d1212b8 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:52.576100 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f9eb76-9588-4ad5-a02a-68710d1212b8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:52.576631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f9eb76-9588-4ad5-a02a-68710d1212b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:52.578454 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f9eb76-9588-4ad5-a02a-68710d1212b8 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:52.600892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 360/703] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Aug 4 04:26:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:52.606522 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c797da-701a-443f-9f6b-173e0a2fd87c None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:52.722573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:52.731472 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c797da-701a-443f-9f6b-173e0a2fd87c None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:52.748202 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3bea4-03ca-45b1-9650-80924b6305f7 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:52.748527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3bea4-03ca-45b1-9650-80924b6305f7 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:52.748756 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3bea4-03ca-45b1-9650-80924b6305f7 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:52.750391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 361/704] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:26:52.757383 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcef521-0531-41ae-bb2d-2f541a226012 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:52.757999 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcef521-0531-41ae-bb2d-2f541a226012 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:52.758420 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcef521-0531-41ae-bb2d-2f541a226012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:52.773642 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a907653-7d02-4a95-a74b-fbc1f98e4adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:52.774287 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a907653-7d02-4a95-a74b-fbc1f98e4adb None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:52.774587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a907653-7d02-4a95-a74b-fbc1f98e4adb None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:52.774844 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a907653-7d02-4a95-a74b-fbc1f98e4adb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:52.785391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-adcef521-0531-41ae-bb2d-2f541a226012 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:52.787825 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-adcef521-0531-41ae-bb2d-2f541a226012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:52.871096 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a907653-7d02-4a95-a74b-fbc1f98e4adb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:52.872863 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a907653-7d02-4a95-a74b-fbc1f98e4adb None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:52.882458 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 362/705] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:52.906776 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcef521-0531-41ae-bb2d-2f541a226012 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:52.940095 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:52.981530 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:52.993659 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 344/706] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 04:26:52.996136 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a907653-7d02-4a95-a74b-fbc1f98e4adb None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:53.021519 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcef521-0531-41ae-bb2d-2f541a226012 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:53.060680 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911287e-f914-41f0-b657-30f262a863e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:53.061364 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911287e-f914-41f0-b657-30f262a863e7 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:53.061912 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911287e-f914-41f0-b657-30f262a863e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:53.062287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911287e-f914-41f0-b657-30f262a863e7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:53.063043 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911287e-f914-41f0-b657-30f262a863e7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:53.065489 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911287e-f914-41f0-b657-30f262a863e7 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:53.077558 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-f911287e-f914-41f0-b657-30f262a863e7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 04:26:53.077558 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:53.077558 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:26:53.077558 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:26:53.077558 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:26:53.077558 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:26:53.077558 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:26:53.077558 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:26:53.077558 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 04:26:53.077558 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:53.077558 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:26:53.077558 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:53.077558 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:53.077558 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:53.077558 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:53.077558 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:53.078783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:53.078783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:53.078783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:53.078783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:53.078783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:53.078783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:53.078783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:53.078783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:26:53.078783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:26:53.078783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:26:53.078783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:26:53.078783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:53.078783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:53.078783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:53.078783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:53.078783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:53.079845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:53.079845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:53.079845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:53.079845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:53.079845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:53.079845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:53.079845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:53.079845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:53.079845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:53.079845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:53.079845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:53.079845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:26:53.079845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:26:53.079845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 04:26:53.079845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:53.079845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 363/707] 10.222.0.47 () {56 vars in 1129 bytes} [Thu Aug 4 04:26:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:53.085244 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911287e-f914-41f0-b657-30f262a863e7 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:53.110177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a907653-7d02-4a95-a74b-fbc1f98e4adb None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:53.203207 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911287e-f914-41f0-b657-30f262a863e7 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:53.205894 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:53.245257 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa813be-098d-4567-84f5-177b034db817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:53.245996 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa813be-098d-4567-84f5-177b034db817 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:53.246338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa813be-098d-4567-84f5-177b034db817 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:53.246673 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa813be-098d-4567-84f5-177b034db817 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:53.247375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa813be-098d-4567-84f5-177b034db817 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:53.247688 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa813be-098d-4567-84f5-177b034db817 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:53.249457 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa813be-098d-4567-84f5-177b034db817 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:53.255262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 364/708] 10.222.0.47 () {56 vars in 1124 bytes} [Thu Aug 4 04:26:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:53.261000 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa813be-098d-4567-84f5-177b034db817 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:53.276183 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbb2e4-1741-4e73-8a46-5e356a361117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:53.276830 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbb2e4-1741-4e73-8a46-5e356a361117 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:53.277159 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbb2e4-1741-4e73-8a46-5e356a361117 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:53.277437 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbb2e4-1741-4e73-8a46-5e356a361117 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:53.289717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fbb2e4-1741-4e73-8a46-5e356a361117 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:53.291302 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fbb2e4-1741-4e73-8a46-5e356a361117 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:53.308529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-93fbb2e4-1741-4e73-8a46-5e356a361117 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:53.308529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:53.308529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:53.308529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:53.308529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:53.308529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:53.308529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:53.308529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:53.308529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:53.308529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:53.308529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:53.308529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:53.308529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:53.308529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:53.308529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:53.309446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:53.309446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:53.309446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:53.309446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:53.309446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:53.309446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:53.309446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:53.309446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:53.309446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:53.309446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:53.309446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:53.309446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:53.309446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:53.309446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:53.309446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:53.309446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:53.309446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:53.310339 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:53.310339 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:53.310339 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:53.310339 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:53.313668 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 345/709] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:53.316440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbb2e4-1741-4e73-8a46-5e356a361117 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:53.372119 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:53.377628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa813be-098d-4567-84f5-177b034db817 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:53.415484 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b94eb-1f14-4d02-a544-86d8d8688bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:53.416357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b94eb-1f14-4d02-a544-86d8d8688bac None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:53.416887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b94eb-1f14-4d02-a544-86d8d8688bac None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:53.417348 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b94eb-1f14-4d02-a544-86d8d8688bac None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:53.418290 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b94eb-1f14-4d02-a544-86d8d8688bac None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:53.418745 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b94eb-1f14-4d02-a544-86d8d8688bac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:53.420707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b94eb-1f14-4d02-a544-86d8d8688bac None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:53.427190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 365/710] 10.222.0.47 () {56 vars in 1124 bytes} [Thu Aug 4 04:26:53 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:53.427377 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbb2e4-1741-4e73-8a46-5e356a361117 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:53.435432 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacc765-d925-43b2-b85a-50c85db6ee9d None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:53.435949 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacc765-d925-43b2-b85a-50c85db6ee9d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:53.436376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacc765-d925-43b2-b85a-50c85db6ee9d None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:53.438300 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 366/711] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:53.444090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525115e-6909-4b4e-952a-842225ae3a02 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:53.444777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525115e-6909-4b4e-952a-842225ae3a02 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:53.445291 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525115e-6909-4b4e-952a-842225ae3a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:53.478809 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33288cb1-5a23-436e-946b-63c59bfcbec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:53.479488 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33288cb1-5a23-436e-946b-63c59bfcbec0 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:53.479769 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33288cb1-5a23-436e-946b-63c59bfcbec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:53.480089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33288cb1-5a23-436e-946b-63c59bfcbec0 None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:53.480807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33288cb1-5a23-436e-946b-63c59bfcbec0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:53.481111 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33288cb1-5a23-436e-946b-63c59bfcbec0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:53.482751 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33288cb1-5a23-436e-946b-63c59bfcbec0 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:53.490294 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-6525115e-6909-4b4e-952a-842225ae3a02 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:53.491945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6525115e-6909-4b4e-952a-842225ae3a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:53.520230 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 346/712] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:53.521584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33288cb1-5a23-436e-946b-63c59bfcbec0 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:53.541973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:53.662456 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 367/713] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:26:53.705234 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525115e-6909-4b4e-952a-842225ae3a02 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:53.710015 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33288cb1-5a23-436e-946b-63c59bfcbec0 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:53.727785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:53.786803 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be9e4b-c649-40c3-a870-2a913116c82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:53.787467 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be9e4b-c649-40c3-a870-2a913116c82b None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:53.787981 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be9e4b-c649-40c3-a870-2a913116c82b None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:53.788407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be9e4b-c649-40c3-a870-2a913116c82b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:53.804375 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be9e4b-c649-40c3-a870-2a913116c82b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:53.806328 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be9e4b-c649-40c3-a870-2a913116c82b None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:53.830318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-f4be9e4b-c649-40c3-a870-2a913116c82b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:53.830318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:53.830318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:53.830318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:53.830318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:53.830318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:53.830318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:53.830318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:53.830318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:53.830318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:53.830318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:53.830318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:53.830318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:26:53.830318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:26:53.830318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:26:53.831567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:26:53.831567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:53.831567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:53.831567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:53.831567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:53.831567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:53.831567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:53.831567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:53.831567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:53.831567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:53.831567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:53.831567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:53.831567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:53.831567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:53.831567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:53.831567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:53.831567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:53.832587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:26:53.832587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:26:53.832587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:26:53.832587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:26:53.832587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:26:53.832587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:26:53.832587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:53.832587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:53.840024 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 347/714] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:26:53.844955 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be9e4b-c649-40c3-a870-2a913116c82b None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:53.900900 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525115e-6909-4b4e-952a-842225ae3a02 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:53.975322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc54b1-bc0d-4e15-bce5-6db208a16e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:53.975977 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc54b1-bc0d-4e15-bce5-6db208a16e42 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:53.976256 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc54b1-bc0d-4e15-bce5-6db208a16e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:53.976528 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc54b1-bc0d-4e15-bce5-6db208a16e42 None admin] PATH_INFO: `/v3/services/network` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:53.977159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc54b1-bc0d-4e15-bce5-6db208a16e42 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:53.978571 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc54b1-bc0d-4e15-bce5-6db208a16e42 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:53.999491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-7afc54b1-bc0d-4e15-bce5-6db208a16e42 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 04:26:53.999491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:53.999491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:26:53.999491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:26:53.999491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:26:53.999491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:26:53.999491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:26:53.999491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:26:53.999491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 04:26:53.999491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:53.999491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:26:53.999491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:53.999491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:53.999491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:53.999491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:53.999491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:53.999491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:54.000608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:54.000608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:54.000608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:54.000608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:54.000608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:54.000608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:54.000608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:26:54.000608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:26:54.000608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:26:54.000608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:26:54.000608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:54.000608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:54.000608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:54.000608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:54.000608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:54.000608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:54.002146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:54.002146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:54.002146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:54.002146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:54.002146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:54.002146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:54.002146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:54.002146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:54.002146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:54.002146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:54.002146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:26:54.002146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:26:54.002146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 04:26:54.002146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:54.013091 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 368/715] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:53 2022] GET /identity/v3/services/network => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:54.019302 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc54b1-bc0d-4e15-bce5-6db208a16e42 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:54.024549 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be9e4b-c649-40c3-a870-2a913116c82b None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:54.066704 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b07b79-a369-427b-8bed-b464924c09cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:54.072239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b07b79-a369-427b-8bed-b464924c09cd None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:54.072585 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b07b79-a369-427b-8bed-b464924c09cd None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:54.072794 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b07b79-a369-427b-8bed-b464924c09cd None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:54.073745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b07b79-a369-427b-8bed-b464924c09cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:54.073952 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:54.074052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b07b79-a369-427b-8bed-b464924c09cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:54.075587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b07b79-a369-427b-8bed-b464924c09cd None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:54.109758 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 348/716] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:26:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:54.122870 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b07b79-a369-427b-8bed-b464924c09cd None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:54.184954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc54b1-bc0d-4e15-bce5-6db208a16e42 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:54.201351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:54.264833 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac2ba5-4808-440b-883b-8d0360e1bc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:54.265327 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac2ba5-4808-440b-883b-8d0360e1bc6c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:54.265697 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac2ba5-4808-440b-883b-8d0360e1bc6c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:54.266042 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac2ba5-4808-440b-883b-8d0360e1bc6c None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:54.266882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac2ba5-4808-440b-883b-8d0360e1bc6c None admin] RBAC: Adding query filter params (name=network) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:54.267706 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac2ba5-4808-440b-883b-8d0360e1bc6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:54.269477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac2ba5-4808-440b-883b-8d0360e1bc6c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:54.281236 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 369/717] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Aug 4 04:26:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:54.286694 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac2ba5-4808-440b-883b-8d0360e1bc6c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:54.287911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b07b79-a369-427b-8bed-b464924c09cd None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:54.338074 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efacf1b0-8fa4-425e-b72a-f63b6b77343a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:54.338977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efacf1b0-8fa4-425e-b72a-f63b6b77343a None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:54.339485 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efacf1b0-8fa4-425e-b72a-f63b6b77343a None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:54.339930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efacf1b0-8fa4-425e-b72a-f63b6b77343a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:54.340929 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efacf1b0-8fa4-425e-b72a-f63b6b77343a None admin] RBAC: Adding query filter params (role.id=4a6517e5890a49718303007dac0756c7, scope.project.id=f37a5f3140ec431da8ea14568654114a, user.id=caa2901451aa4554bdfcac1bddbc620a) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:54.341426 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efacf1b0-8fa4-425e-b72a-f63b6b77343a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:54.343065 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efacf1b0-8fa4-425e-b72a-f63b6b77343a None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:54.354321 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 349/718] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:26:54 2022] GET /identity/v3/role_assignments?user.id=caa2901451aa4554bdfcac1bddbc620a&scope.project.id=f37a5f3140ec431da8ea14568654114a&role.id=4a6517e5890a49718303007dac0756c7 => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:54.409938 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:54.438005 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac2ba5-4808-440b-883b-8d0360e1bc6c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:54.480333 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62b56a-2e65-441e-9050-ddcef2c84cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:54.480982 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62b56a-2e65-441e-9050-ddcef2c84cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:54.481269 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62b56a-2e65-441e-9050-ddcef2c84cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:54.481587 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62b56a-2e65-441e-9050-ddcef2c84cd8 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:54.482273 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b62b56a-2e65-441e-9050-ddcef2c84cd8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:54.482582 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b62b56a-2e65-441e-9050-ddcef2c84cd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:54.483991 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b62b56a-2e65-441e-9050-ddcef2c84cd8 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:54.492519 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 370/719] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Aug 4 04:26:54 2022] GET /identity/v3/services?type=network => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:55.059257 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7afc8-ca22-4772-8862-9b1a761537f7 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:55.059774 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7afc8-ca22-4772-8862-9b1a761537f7 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:55.059774 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7afc8-ca22-4772-8862-9b1a761537f7 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:55.061317 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 350/720] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:55.067136 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e00380-4181-499d-93a2-6b364bb15bc5 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:55.067689 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e00380-4181-499d-93a2-6b364bb15bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:55.067968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e00380-4181-499d-93a2-6b364bb15bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:55.096178 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-a8e00380-4181-499d-93a2-6b364bb15bc5 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:55.096178 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a8e00380-4181-499d-93a2-6b364bb15bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:55.190118 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 371/721] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 04:26:55.233223 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7afc8-ca22-4772-8862-9b1a761537f7 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:55.379031 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:55.389391 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7afc8-ca22-4772-8862-9b1a761537f7 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:55.420728 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09e687-3c88-4d4b-b553-c1fa941bba16 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:55.421037 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09e687-3c88-4d4b-b553-c1fa941bba16 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:55.421246 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09e687-3c88-4d4b-b553-c1fa941bba16 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:55.423018 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 372/722] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:55.430159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b1b61-1725-474f-a5d4-7cac4f438ec8 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:55.430443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b1b61-1725-474f-a5d4-7cac4f438ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:55.430719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b1b61-1725-474f-a5d4-7cac4f438ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:55.445236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbad830-a562-4f15-8b6c-16d0c8e90615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:55.446158 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbad830-a562-4f15-8b6c-16d0c8e90615 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:55.446586 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbad830-a562-4f15-8b6c-16d0c8e90615 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:55.447000 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbad830-a562-4f15-8b6c-16d0c8e90615 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:55.459373 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbad830-a562-4f15-8b6c-16d0c8e90615 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:55.465091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-7bbad830-a562-4f15-8b6c-16d0c8e90615 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:26:55.465091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:55.465091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:55.465091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:55.465091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:55.465091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:55.465091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:55.465091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:55.465091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:55.465091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:55.465091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:55.465091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:55.465091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:55.465091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:55.465091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:55.465091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:55.466236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:55.466236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:55.466236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:55.466236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:55.466236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:55.466236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:55.466236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:55.466236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:55.466236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:55.466236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:55.466236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:55.466236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:55.466236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:55.466236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:55.466236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:55.466236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:55.467269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:55.467269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:55.467269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:55.467269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:55.467269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:55.467269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:55.467269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:55.467269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:55.467269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:55.467269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:55.467269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:26:55.467269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:55.469763 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 351/723] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:26:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:55.472145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-1d7b1b61-1725-474f-a5d4-7cac4f438ec8 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:55.473355 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1d7b1b61-1725-474f-a5d4-7cac4f438ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:55.482606 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbad830-a562-4f15-8b6c-16d0c8e90615 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:55.612792 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 373/724] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:55.649172 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b1b61-1725-474f-a5d4-7cac4f438ec8 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:55.650602 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbad830-a562-4f15-8b6c-16d0c8e90615 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:55.654656 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:55.692688 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d45c9-be18-4055-9306-a416d97ac801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:55.693314 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d45c9-be18-4055-9306-a416d97ac801 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:55.693614 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d45c9-be18-4055-9306-a416d97ac801 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:55.693943 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d45c9-be18-4055-9306-a416d97ac801 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:55.694672 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d45c9-be18-4055-9306-a416d97ac801 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:55.694971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d45c9-be18-4055-9306-a416d97ac801 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:55.696333 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d45c9-be18-4055-9306-a416d97ac801 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:55.710683 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 352/725] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:55.716390 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d45c9-be18-4055-9306-a416d97ac801 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:55.734399 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:55.828266 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d45c9-be18-4055-9306-a416d97ac801 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:55.868125 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50e30a-1f2a-4bc6-ab75-4232e50af025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:55.868780 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50e30a-1f2a-4bc6-ab75-4232e50af025 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:55.869061 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50e30a-1f2a-4bc6-ab75-4232e50af025 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:55.869333 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50e30a-1f2a-4bc6-ab75-4232e50af025 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:55.880123 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b1b61-1725-474f-a5d4-7cac4f438ec8 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:55.881536 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50e30a-1f2a-4bc6-ab75-4232e50af025 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:55.883008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50e30a-1f2a-4bc6-ab75-4232e50af025 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:55.900454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-7c50e30a-1f2a-4bc6-ab75-4232e50af025 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:55.900454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:55.900454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:55.900454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:55.900454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:55.900454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:55.900454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:55.900454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:55.900454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:55.900454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:55.900454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:55.900454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:55.900454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:55.900454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:55.900454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:55.901471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:55.901471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:55.901471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:55.901471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:55.901471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:55.901471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:55.901471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:55.901471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:55.901471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:55.901471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:55.901471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:55.901471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:55.901471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:55.901471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:55.901471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:55.901471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:55.901471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:55.902384 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:55.902384 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:55.902384 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:55.902384 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:55.902613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 353/726] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:26:55.909356 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50e30a-1f2a-4bc6-ab75-4232e50af025 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:55.919495 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5eba72-15fc-4440-84d6-5a81c4e42d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:55.920160 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5eba72-15fc-4440-84d6-5a81c4e42d7c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:55.920717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5eba72-15fc-4440-84d6-5a81c4e42d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:55.920989 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5eba72-15fc-4440-84d6-5a81c4e42d7c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:55.932915 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5eba72-15fc-4440-84d6-5a81c4e42d7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:55.937933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-5e5eba72-15fc-4440-84d6-5a81c4e42d7c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:55.937933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:55.937933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:55.937933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:55.937933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:55.937933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:55.937933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:55.937933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:55.937933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:55.937933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:55.937933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:55.937933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:55.937933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:55.937933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:55.937933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:55.937933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:55.938939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:55.938939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:55.938939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:55.938939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:55.938939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:55.938939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:55.938939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:55.938939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:55.938939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:55.938939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:55.938939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:55.938939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:55.938939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:55.938939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:55.938939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:55.938939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:55.939757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:55.939757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:55.939757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:55.939757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:55.939757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:55.939757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:55.939757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:55.939757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:55.939757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:55.939757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:55.939757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:26:55.939757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:55.939757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 374/727] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Aug 4 04:26:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:26:55.947435 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5eba72-15fc-4440-84d6-5a81c4e42d7c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:56.020507 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:56.057097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:56.071138 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50e30a-1f2a-4bc6-ab75-4232e50af025 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:56.102262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5eba72-15fc-4440-84d6-5a81c4e42d7c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:56.120254 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69611669-2a8c-4c3c-94d2-75aa01c0734f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:56.121303 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69611669-2a8c-4c3c-94d2-75aa01c0734f None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:56.121641 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69611669-2a8c-4c3c-94d2-75aa01c0734f None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:56.121922 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69611669-2a8c-4c3c-94d2-75aa01c0734f None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:56.122966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69611669-2a8c-4c3c-94d2-75aa01c0734f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:56.123279 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69611669-2a8c-4c3c-94d2-75aa01c0734f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:56.124825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69611669-2a8c-4c3c-94d2-75aa01c0734f None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:56.141136 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 354/728] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:26:56.144052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2a6e6-a015-4fce-ba47-ceb99ecaad9e None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:56.144530 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2a6e6-a015-4fce-ba47-ceb99ecaad9e None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:56.145059 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2a6e6-a015-4fce-ba47-ceb99ecaad9e None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:56.151172 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 355/729] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:56.151172 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2a6e6-a015-4fce-ba47-ceb99ecaad9e None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:56.155547 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d556fc-9179-4b48-b2d8-de70926dc584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:56.156460 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d556fc-9179-4b48-b2d8-de70926dc584 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:56.156970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d556fc-9179-4b48-b2d8-de70926dc584 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:56.157507 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d556fc-9179-4b48-b2d8-de70926dc584 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:56.158568 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d556fc-9179-4b48-b2d8-de70926dc584 None admin] RBAC: Adding query filter params (name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:56.159077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d556fc-9179-4b48-b2d8-de70926dc584 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:56.160679 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d556fc-9179-4b48-b2d8-de70926dc584 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:56.170122 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 375/730] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Aug 4 04:26:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:56.171957 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604ea3b-02c9-4269-af16-b7b924639627 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:56.172449 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604ea3b-02c9-4269-af16-b7b924639627 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:56.172924 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604ea3b-02c9-4269-af16-b7b924639627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:56.192439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-c604ea3b-02c9-4269-af16-b7b924639627 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:56.194148 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c604ea3b-02c9-4269-af16-b7b924639627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:56.275330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2a6e6-a015-4fce-ba47-ceb99ecaad9e None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:56.293568 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 376/731] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:56.299384 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604ea3b-02c9-4269-af16-b7b924639627 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:56.311437 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:56.324539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3997482-c87e-4c42-99cb-d95617316003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:56.324539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3997482-c87e-4c42-99cb-d95617316003 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:56.324539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3997482-c87e-4c42-99cb-d95617316003 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:56.324897 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3997482-c87e-4c42-99cb-d95617316003 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:56.334577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3997482-c87e-4c42-99cb-d95617316003 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:56.337384 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3997482-c87e-4c42-99cb-d95617316003 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:56.355573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-c3997482-c87e-4c42-99cb-d95617316003 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 04:26:56.355573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:56.355573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:56.355573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:56.355573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:56.355573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:56.355573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:56.355573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:56.355573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:56.355573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:56.355573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:56.355573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:56.355573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:26:56.355573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:26:56.355573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:26:56.355573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:26:56.356807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:56.356807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:56.356807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:26:56.356807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:56.356807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:26:56.356807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:56.356807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:56.356807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:56.356807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:56.356807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:56.356807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:56.356807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:56.356807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:56.356807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:56.356807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:56.356807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:56.356807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:56.356807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:56.358277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:56.358277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:56.358277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:26:56.358277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:26:56.358277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:26:56.358277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:26:56.358277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:26:56.358277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:26:56.358277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 04:26:56.358277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:56.358277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 356/732] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:26:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:56.362334 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3997482-c87e-4c42-99cb-d95617316003 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:56.390458 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:56.442207 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604ea3b-02c9-4269-af16-b7b924639627 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:56.480295 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae04de-6a17-4dd9-8a89-46256569b847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:56.481301 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae04de-6a17-4dd9-8a89-46256569b847 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:56.481898 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae04de-6a17-4dd9-8a89-46256569b847 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:56.482379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae04de-6a17-4dd9-8a89-46256569b847 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:56.491795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3997482-c87e-4c42-99cb-d95617316003 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:56.494543 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ae04de-6a17-4dd9-8a89-46256569b847 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:56.496261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ae04de-6a17-4dd9-8a89-46256569b847 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:56.513809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-33ae04de-6a17-4dd9-8a89-46256569b847 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:56.513809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:56.513809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:56.513809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:56.513809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:56.513809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:56.513809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:56.513809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:56.513809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:56.513809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:56.513809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:56.513809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:56.513809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:56.513809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:56.513809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:56.514754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:56.514754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:56.514754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:56.514754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:56.514754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:56.514754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:56.514754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:56.514754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:56.514754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:56.514754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:56.514754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:56.514754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:56.514754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:56.514754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:56.514754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:56.514754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:56.514754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:56.515620 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:56.515620 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:56.515620 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:56.515620 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:56.518544 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 377/733] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:56.519805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae04de-6a17-4dd9-8a89-46256569b847 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:56.544332 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc773232-8ec6-42c5-a61b-893420d2d425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:56.544977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc773232-8ec6-42c5-a61b-893420d2d425 None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:56.545525 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc773232-8ec6-42c5-a61b-893420d2d425 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:56.545861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc773232-8ec6-42c5-a61b-893420d2d425 None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:56.546591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc773232-8ec6-42c5-a61b-893420d2d425 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:56.548671 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc773232-8ec6-42c5-a61b-893420d2d425 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:56.573898 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 357/734] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Aug 4 04:26:56 2022] POST /identity/v3/services => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:26:56.573898 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc773232-8ec6-42c5-a61b-893420d2d425 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:56.688010 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:56.751601 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae04de-6a17-4dd9-8a89-46256569b847 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:56.769235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:56.812777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75fb290-10a0-4f0c-9ce5-490ac5857410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:56.813427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fb290-10a0-4f0c-9ce5-490ac5857410 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:56.813784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fb290-10a0-4f0c-9ce5-490ac5857410 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:56.814066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fb290-10a0-4f0c-9ce5-490ac5857410 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:56.814403 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc773232-8ec6-42c5-a61b-893420d2d425 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:56.814882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75fb290-10a0-4f0c-9ce5-490ac5857410 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:56.815190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75fb290-10a0-4f0c-9ce5-490ac5857410 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:56.816832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75fb290-10a0-4f0c-9ce5-490ac5857410 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:56.838868 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 378/735] 10.222.0.47 () {56 vars in 1135 bytes} [Thu Aug 4 04:26:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:56.844774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75fb290-10a0-4f0c-9ce5-490ac5857410 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:56.863376 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01749740-4103-48c8-906e-02461ddd8f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:56.864621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01749740-4103-48c8-906e-02461ddd8f93 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:56.865099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01749740-4103-48c8-906e-02461ddd8f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:56.865612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01749740-4103-48c8-906e-02461ddd8f93 None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:56.866621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01749740-4103-48c8-906e-02461ddd8f93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:56.867134 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01749740-4103-48c8-906e-02461ddd8f93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:56.868904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01749740-4103-48c8-906e-02461ddd8f93 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:56.885312 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 358/736] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:56.891787 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01749740-4103-48c8-906e-02461ddd8f93 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:57.025017 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01749740-4103-48c8-906e-02461ddd8f93 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:57.027136 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:57.068426 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521b732-5859-4bca-a33a-1a61d0113f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:57.069838 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521b732-5859-4bca-a33a-1a61d0113f9b None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:57.070327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521b732-5859-4bca-a33a-1a61d0113f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:57.070799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521b732-5859-4bca-a33a-1a61d0113f9b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:57.080946 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c521b732-5859-4bca-a33a-1a61d0113f9b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:57.083288 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c521b732-5859-4bca-a33a-1a61d0113f9b None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:57.096337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-c521b732-5859-4bca-a33a-1a61d0113f9b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 04:26:57.096337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:57.096337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:57.096337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:57.096337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:57.096337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:57.096337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:57.096337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:57.096337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:57.096337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:57.096337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:57.096337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:57.096337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:26:57.096337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:26:57.096337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:26:57.096337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:26:57.097309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:57.097309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:57.097309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:26:57.097309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:57.097309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:26:57.097309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:57.097309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:57.097309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:57.097309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:57.097309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:57.097309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:57.097309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:57.097309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:57.097309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:57.097309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:57.097309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:57.097309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:57.097309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:57.098259 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:57.098259 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:57.098259 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:26:57.098259 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:26:57.098259 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:26:57.098259 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:26:57.098259 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:26:57.098259 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:26:57.098259 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 04:26:57.098259 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:57.101399 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 359/737] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Aug 4 04:26:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:57.108974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ad614-985c-490b-b164-9108670621b7 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:57.109795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ad614-985c-490b-b164-9108670621b7 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:57.110291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ad614-985c-490b-b164-9108670621b7 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:57.110680 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75fb290-10a0-4f0c-9ce5-490ac5857410 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:57.112714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 360/738] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:57 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:57.113864 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ad614-985c-490b-b164-9108670621b7 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:57.158046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa78be-541c-4ec6-ac08-3d4350499d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:57.158887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa78be-541c-4ec6-ac08-3d4350499d80 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:57.159171 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa78be-541c-4ec6-ac08-3d4350499d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:57.159444 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa78be-541c-4ec6-ac08-3d4350499d80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:57.193372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:57.242428 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa78be-541c-4ec6-ac08-3d4350499d80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:57.243095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ad614-985c-490b-b164-9108670621b7 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:57.243908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa78be-541c-4ec6-ac08-3d4350499d80 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:57.283589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede54bd-686d-44b1-af4c-cc79ddbfd314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:57.284207 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede54bd-686d-44b1-af4c-cc79ddbfd314 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:57.284481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede54bd-686d-44b1-af4c-cc79ddbfd314 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:57.284752 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede54bd-686d-44b1-af4c-cc79ddbfd314 None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:57.285576 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede54bd-686d-44b1-af4c-cc79ddbfd314 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:57.285874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede54bd-686d-44b1-af4c-cc79ddbfd314 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:57.287403 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede54bd-686d-44b1-af4c-cc79ddbfd314 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:57.309268 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 361/739] 10.222.0.47 () {56 vars in 1149 bytes} [Thu Aug 4 04:26:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:57.310853 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb82acd-a98d-48e5-91c7-9a94ef468bdb None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:57.311130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb82acd-a98d-48e5-91c7-9a94ef468bdb None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:57.311396 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb82acd-a98d-48e5-91c7-9a94ef468bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:57.330076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-4fb82acd-a98d-48e5-91c7-9a94ef468bdb None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:57.331262 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-4fb82acd-a98d-48e5-91c7-9a94ef468bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:57.346546 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:57.355555 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 379/740] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:56 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:26:57.355555 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587be79f-5f10-49ad-9fa2-a51e5d1e63fb None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:57.355555 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587be79f-5f10-49ad-9fa2-a51e5d1e63fb None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:57.355555 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587be79f-5f10-49ad-9fa2-a51e5d1e63fb None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:57.358710 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 380/741] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:57.362610 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587be79f-5f10-49ad-9fa2-a51e5d1e63fb None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:57.408285 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 362/742] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:57 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:57.409230 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb82acd-a98d-48e5-91c7-9a94ef468bdb None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:57.451630 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:57.494171 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587be79f-5f10-49ad-9fa2-a51e5d1e63fb None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:57.527256 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb82acd-a98d-48e5-91c7-9a94ef468bdb None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:57.530468 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d7149-29cb-4454-bb4b-6401e3ca1be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:57.531078 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d7149-29cb-4454-bb4b-6401e3ca1be6 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:57.531355 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d7149-29cb-4454-bb4b-6401e3ca1be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:57.531623 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d7149-29cb-4454-bb4b-6401e3ca1be6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:57.564582 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa5577-dedf-47fb-8d9c-8e3d08f69bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:57.565143 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa5577-dedf-47fb-8d9c-8e3d08f69bbe None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:57.565426 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa5577-dedf-47fb-8d9c-8e3d08f69bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:57.565736 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa5577-dedf-47fb-8d9c-8e3d08f69bbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:57.575792 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa5577-dedf-47fb-8d9c-8e3d08f69bbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:57.577243 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa5577-dedf-47fb-8d9c-8e3d08f69bbe None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:57.592564 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-cbfa5577-dedf-47fb-8d9c-8e3d08f69bbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:57.592564 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:57.592564 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:57.592564 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:57.592564 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:57.592564 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:57.592564 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:57.592564 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:57.592564 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:57.592564 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:57.592564 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:57.592564 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:57.592564 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:57.592564 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:57.592564 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:57.593486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:57.593486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:57.593486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:57.593486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:57.593486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:57.593486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:57.593486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:57.593486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:57.593486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:57.593486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:57.593486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:57.593486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:57.593486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:57.593486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:57.593486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:57.593486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:57.593486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:57.594414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:57.594414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:57.594414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:57.594414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:57.594414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 363/743] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:57.595858 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1869ce1-6dd1-43dd-a62e-c3cdba300d2f None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:57.596193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1869ce1-6dd1-43dd-a62e-c3cdba300d2f None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:57.596485 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1869ce1-6dd1-43dd-a62e-c3cdba300d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:57.603216 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:57.612205 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d7149-29cb-4454-bb4b-6401e3ca1be6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:57.613683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d7149-29cb-4454-bb4b-6401e3ca1be6 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:57.615757 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-b1869ce1-6dd1-43dd-a62e-c3cdba300d2f None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:57.616953 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-b1869ce1-6dd1-43dd-a62e-c3cdba300d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:57.677259 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:57.685859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 364/744] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:57 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 04:26:57.686693 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1869ce1-6dd1-43dd-a62e-c3cdba300d2f None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:57.735339 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 381/745] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:26:57 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 04:26:57.736853 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d7149-29cb-4454-bb4b-6401e3ca1be6 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:57.805988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1869ce1-6dd1-43dd-a62e-c3cdba300d2f None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:57.838782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:57.844155 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d7149-29cb-4454-bb4b-6401e3ca1be6 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:57.845212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8858c0-c7a0-4900-bdd4-220aad0981c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:57.846153 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8858c0-c7a0-4900-bdd4-220aad0981c6 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:57.846430 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8858c0-c7a0-4900-bdd4-220aad0981c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:57.847479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8858c0-c7a0-4900-bdd4-220aad0981c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:57.859886 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8858c0-c7a0-4900-bdd4-220aad0981c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:26:57.865815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-ea8858c0-c7a0-4900-bdd4-220aad0981c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:26:57.865815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:57.865815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:57.865815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:57.865815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:57.865815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:57.865815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:57.865815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:57.865815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:57.865815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:57.865815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:57.865815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:57.865815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:26:57.865815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:26:57.865815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:26:57.865815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:26:57.866714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:26:57.866714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:26:57.866714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:26:57.866714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:26:57.866714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:57.866714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:57.866714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:57.866714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:57.866714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:57.866714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:57.866714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:57.866714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:57.866714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:57.866714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:57.866714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:57.866714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:57.867546 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:57.867546 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:57.867546 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:57.867546 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:57.867546 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:26:57.867546 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:26:57.867546 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:26:57.867546 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:26:57.867546 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:26:57.867546 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:26:57.867546 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:26:57.867546 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:57.867546 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 365/746] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:26:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:57.868229 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8858c0-c7a0-4900-bdd4-220aad0981c6 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:57.887075 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7720a25-5a1e-43a6-8872-0c43fc08b3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:57.887698 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7720a25-5a1e-43a6-8872-0c43fc08b3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:57.887979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7720a25-5a1e-43a6-8872-0c43fc08b3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:57.888251 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7720a25-5a1e-43a6-8872-0c43fc08b3e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:57.888998 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7720a25-5a1e-43a6-8872-0c43fc08b3e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:57.889300 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7720a25-5a1e-43a6-8872-0c43fc08b3e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:57.890794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7720a25-5a1e-43a6-8872-0c43fc08b3e2 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:57.906892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 382/747] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:57.908185 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7720a25-5a1e-43a6-8872-0c43fc08b3e2 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:57.938181 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:57.993846 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8858c0-c7a0-4900-bdd4-220aad0981c6 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:58.026203 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7720a25-5a1e-43a6-8872-0c43fc08b3e2 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:58.030536 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c003b-7425-4809-8c62-791e854a5563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:58.031219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c003b-7425-4809-8c62-791e854a5563 None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:58.031506 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c003b-7425-4809-8c62-791e854a5563 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:58.031781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c003b-7425-4809-8c62-791e854a5563 None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:58.032625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89c003b-7425-4809-8c62-791e854a5563 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:58.034596 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89c003b-7425-4809-8c62-791e854a5563 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:58.051571 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 366/748] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Aug 4 04:26:57 2022] POST /identity/v3/services => generated 240 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:26:58.052369 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c003b-7425-4809-8c62-791e854a5563 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:58.081375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2eb2e-cef5-46f7-a552-50d27978c625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:58.082776 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2eb2e-cef5-46f7-a552-50d27978c625 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:58.083267 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2eb2e-cef5-46f7-a552-50d27978c625 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:58.084012 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2eb2e-cef5-46f7-a552-50d27978c625 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:58.102051 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2eb2e-cef5-46f7-a552-50d27978c625 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:58.104560 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2eb2e-cef5-46f7-a552-50d27978c625 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:58.116923 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:58.130719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-e2c2eb2e-cef5-46f7-a552-50d27978c625 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:58.130719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:58.130719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:58.130719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:58.130719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:58.130719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:58.130719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:58.130719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:58.130719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:58.130719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:58.130719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:58.130719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:58.130719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:58.130719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:58.130719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:58.132034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:58.132034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:58.132034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:58.132034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:58.132034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:58.132034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:58.132034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:58.132034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:58.132034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:58.132034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:58.132034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:58.132034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:58.132034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:58.132034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:58.132034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:58.132034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:58.132034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:58.132915 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:58.132915 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:58.132915 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:58.132915 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:58.140628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 383/749] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:58.140628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2eb2e-cef5-46f7-a552-50d27978c625 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:58.189238 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c003b-7425-4809-8c62-791e854a5563 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:58.230971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6727cf-f035-4a5e-a3be-3675e6cb1a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:58.231634 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6727cf-f035-4a5e-a3be-3675e6cb1a9e None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:58.231917 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6727cf-f035-4a5e-a3be-3675e6cb1a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:58.232189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6727cf-f035-4a5e-a3be-3675e6cb1a9e None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:58.232908 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6727cf-f035-4a5e-a3be-3675e6cb1a9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:58.233212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6727cf-f035-4a5e-a3be-3675e6cb1a9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:58.234292 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:58.234688 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6727cf-f035-4a5e-a3be-3675e6cb1a9e None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:58.244944 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 367/750] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:26:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:58.245847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6727cf-f035-4a5e-a3be-3675e6cb1a9e None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:58.278075 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2eb2e-cef5-46f7-a552-50d27978c625 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:58.318337 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f5fcf-da67-477f-a56f-a2dbaa48d5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:58.318982 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f5fcf-da67-477f-a56f-a2dbaa48d5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:58.319267 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f5fcf-da67-477f-a56f-a2dbaa48d5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:58.319556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f5fcf-da67-477f-a56f-a2dbaa48d5f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:58.326643 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f5fcf-da67-477f-a56f-a2dbaa48d5f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:58.328096 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f5fcf-da67-477f-a56f-a2dbaa48d5f9 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:58.340467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-6e8f5fcf-da67-477f-a56f-a2dbaa48d5f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:58.340467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:58.340467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:58.340467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:58.340467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:58.340467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:58.340467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:58.340467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:58.340467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:58.340467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:58.340467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:58.340467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:58.340467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:26:58.340467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:26:58.340467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:26:58.341378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:26:58.341378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:58.341378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:58.341378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:58.341378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:58.341378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:58.341378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:58.341378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:58.341378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:58.341378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:58.341378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:58.341378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:58.341378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:58.341378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:58.341378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:58.341378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:58.341378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:58.342264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:26:58.342264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:26:58.342264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:26:58.342264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:26:58.342264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:26:58.342264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:26:58.342264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:58.342264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:58.342264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 384/751] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:58.342734 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f5fcf-da67-477f-a56f-a2dbaa48d5f9 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:58.370279 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6727cf-f035-4a5e-a3be-3675e6cb1a9e None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:58.401963 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:58.413054 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6443a35-c162-430c-a883-54cb8d7498ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:58.413719 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6443a35-c162-430c-a883-54cb8d7498ae None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:58.414009 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6443a35-c162-430c-a883-54cb8d7498ae None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:58.414293 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6443a35-c162-430c-a883-54cb8d7498ae None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:58.415579 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6443a35-c162-430c-a883-54cb8d7498ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:58.415893 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6443a35-c162-430c-a883-54cb8d7498ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:58.417866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6443a35-c162-430c-a883-54cb8d7498ae None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:58.441430 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 368/752] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:58.442412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6443a35-c162-430c-a883-54cb8d7498ae None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:58.472101 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f5fcf-da67-477f-a56f-a2dbaa48d5f9 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:58.520644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:58.529479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d1a672-61ac-4645-ac6a-9ad46faa7e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:58.530799 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1a672-61ac-4645-ac6a-9ad46faa7e36 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:58.531564 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1a672-61ac-4645-ac6a-9ad46faa7e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:58.532059 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1a672-61ac-4645-ac6a-9ad46faa7e36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:58.547468 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d1a672-61ac-4645-ac6a-9ad46faa7e36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:58.550022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d1a672-61ac-4645-ac6a-9ad46faa7e36 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:58.570669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-67d1a672-61ac-4645-ac6a-9ad46faa7e36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:58.570669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:58.570669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:58.570669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:58.570669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:58.570669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:58.570669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:58.570669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:58.570669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:58.570669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:58.570669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:58.570669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:58.570669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:58.570669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:58.570669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:58.571632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:58.571632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:58.571632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:58.571632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:58.571632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:58.571632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:58.571632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:58.571632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:58.571632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:58.571632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:58.571632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:58.571632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:58.571632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:58.571632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:58.571632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:58.571632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:58.571632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:58.572526 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:58.572526 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:58.572526 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:58.572526 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:58.574639 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 385/753] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:58.575620 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d1a672-61ac-4645-ac6a-9ad46faa7e36 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:58.575735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6443a35-c162-430c-a883-54cb8d7498ae None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:58.614762 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdeac89-d8fa-470d-a3ad-205858721f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:58.615404 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdeac89-d8fa-470d-a3ad-205858721f3d None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:58.615669 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdeac89-d8fa-470d-a3ad-205858721f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:58.615947 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdeac89-d8fa-470d-a3ad-205858721f3d None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:58.616706 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdeac89-d8fa-470d-a3ad-205858721f3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:58.617018 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdeac89-d8fa-470d-a3ad-205858721f3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:58.618499 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdeac89-d8fa-470d-a3ad-205858721f3d None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:58.632651 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 369/754] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:26:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:58.633496 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdeac89-d8fa-470d-a3ad-205858721f3d None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:58.675599 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:58.703877 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d1a672-61ac-4645-ac6a-9ad46faa7e36 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:58.740040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc10819-3073-4f16-aac0-fb20e82e02e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:58.740787 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc10819-3073-4f16-aac0-fb20e82e02e6 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:58.741063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc10819-3073-4f16-aac0-fb20e82e02e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:58.741346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc10819-3073-4f16-aac0-fb20e82e02e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:58.747969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdeac89-d8fa-470d-a3ad-205858721f3d None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:58.748542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc10819-3073-4f16-aac0-fb20e82e02e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:58.750146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc10819-3073-4f16-aac0-fb20e82e02e6 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:58.762951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-8fc10819-3073-4f16-aac0-fb20e82e02e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:58.762951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:58.762951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:58.762951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:58.762951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:58.762951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:58.762951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:58.762951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:58.762951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:58.762951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:58.762951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:58.762951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:58.762951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:26:58.762951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:26:58.762951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:26:58.764060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:26:58.764060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:58.764060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:58.764060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:58.764060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:58.764060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:58.764060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:58.764060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:58.764060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:58.764060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:58.764060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:58.764060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:58.764060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:58.764060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:58.764060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:58.764060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:58.764060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:58.764983 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:26:58.764983 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:26:58.764983 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:26:58.764983 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:26:58.764983 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:26:58.764983 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:26:58.764983 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:26:58.764983 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:58.769001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 386/755] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:26:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:58.769911 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc10819-3073-4f16-aac0-fb20e82e02e6 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:58.788289 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b9696-bb91-44c2-ba4b-93ec9cea21db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:58.789306 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b9696-bb91-44c2-ba4b-93ec9cea21db None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:58.789798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b9696-bb91-44c2-ba4b-93ec9cea21db None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:58.790298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b9696-bb91-44c2-ba4b-93ec9cea21db None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:58.791180 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b9696-bb91-44c2-ba4b-93ec9cea21db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:58.791807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b9696-bb91-44c2-ba4b-93ec9cea21db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:58.793407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b9696-bb91-44c2-ba4b-93ec9cea21db None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:58.807429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 370/756] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:58.808559 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b9696-bb91-44c2-ba4b-93ec9cea21db None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:58.828747 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:58.880019 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc10819-3073-4f16-aac0-fb20e82e02e6 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:58.913761 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:58.914149 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b12775-9328-413e-8dcc-145e143afd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:58.914760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b12775-9328-413e-8dcc-145e143afd42 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:58.915039 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b12775-9328-413e-8dcc-145e143afd42 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:58.915310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b12775-9328-413e-8dcc-145e143afd42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:58.916097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b12775-9328-413e-8dcc-145e143afd42 None admin] RBAC: Adding query filter params (role.id=4a6517e5890a49718303007dac0756c7, scope.project.id=f37a5f3140ec431da8ea14568654114a, user.id=b1e539006166434790823b839bd745d5) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:58.916408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b12775-9328-413e-8dcc-145e143afd42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:58.917870 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b12775-9328-413e-8dcc-145e143afd42 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:58.922017 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b9696-bb91-44c2-ba4b-93ec9cea21db None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:58.933576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 387/757] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:26:58 2022] GET /identity/v3/role_assignments?user.id=b1e539006166434790823b839bd745d5&scope.project.id=f37a5f3140ec431da8ea14568654114a&role.id=4a6517e5890a49718303007dac0756c7 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:58.933576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b12775-9328-413e-8dcc-145e143afd42 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:58.979008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ff43c-1b13-4cd0-903d-655f27347742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:58.979802 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ff43c-1b13-4cd0-903d-655f27347742 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:58.980183 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ff43c-1b13-4cd0-903d-655f27347742 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:58.980553 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ff43c-1b13-4cd0-903d-655f27347742 None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:58.981443 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ff43c-1b13-4cd0-903d-655f27347742 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:58.981879 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ff43c-1b13-4cd0-903d-655f27347742 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:58.983456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ff43c-1b13-4cd0-903d-655f27347742 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:59.005176 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 371/758] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:26:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:59.006825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3e966-0934-4c92-ad74-3639bbca2561 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:59.007218 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3e966-0934-4c92-ad74-3639bbca2561 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:59.007611 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3e966-0934-4c92-ad74-3639bbca2561 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:59.013569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 372/759] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:26:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:26:59.014625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3e966-0934-4c92-ad74-3639bbca2561 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:59.060757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b12775-9328-413e-8dcc-145e143afd42 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:59.071437 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:59.108548 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234070a8-6938-43f0-97ac-5879a2ccdee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:59.109450 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234070a8-6938-43f0-97ac-5879a2ccdee3 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:59.110050 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234070a8-6938-43f0-97ac-5879a2ccdee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:59.110637 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234070a8-6938-43f0-97ac-5879a2ccdee3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:59.119405 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234070a8-6938-43f0-97ac-5879a2ccdee3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:59.121090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234070a8-6938-43f0-97ac-5879a2ccdee3 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:59.134170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-234070a8-6938-43f0-97ac-5879a2ccdee3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 04:26:59.134170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:59.134170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:59.134170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:59.134170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:59.134170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:59.134170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:59.134170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:59.134170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:59.134170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:59.134170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:59.134170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:59.134170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:26:59.134170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:26:59.134170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:26:59.134170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:26:59.135567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:59.135567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:59.135567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:26:59.135567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:59.135567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:26:59.135567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:26:59.135567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:59.135567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:59.135567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:59.135567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:59.135567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:59.135567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:59.135567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:59.135567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:59.135567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:59.135567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:59.135567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:59.135567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:59.136482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:59.136482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:59.136482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:26:59.136482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:26:59.136482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:26:59.136482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:26:59.136482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:26:59.136482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:26:59.136482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 04:26:59.136482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:59.139096 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 388/760] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Aug 4 04:26:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:26:59.140483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171f7a0-860f-4a07-aebd-dde2dc399e38 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:59.140969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171f7a0-860f-4a07-aebd-dde2dc399e38 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:59.141425 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171f7a0-860f-4a07-aebd-dde2dc399e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:59.159696 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3e966-0934-4c92-ad74-3639bbca2561 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:59.162048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b171f7a0-860f-4a07-aebd-dde2dc399e38 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:26:59.163437 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b171f7a0-860f-4a07-aebd-dde2dc399e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:26:59.206269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b9328-9fb9-4c01-bb65-9847497ab162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:59.206899 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b9328-9fb9-4c01-bb65-9847497ab162 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:59.207250 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b9328-9fb9-4c01-bb65-9847497ab162 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:59.207618 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b9328-9fb9-4c01-bb65-9847497ab162 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:59.209044 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b9328-9fb9-4c01-bb65-9847497ab162 None admin] RBAC: Adding query filter params (role.id=cc86b64693b6417e90ad9361460574b0, scope.project.id=f37a5f3140ec431da8ea14568654114a, user.id=3a721708c7e443e799349add27de8f84) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:59.209413 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b9328-9fb9-4c01-bb65-9847497ab162 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:59.210930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b9328-9fb9-4c01-bb65-9847497ab162 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:59.219475 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 373/761] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:26:59 2022] GET /identity/v3/role_assignments?user.id=3a721708c7e443e799349add27de8f84&scope.project.id=f37a5f3140ec431da8ea14568654114a&role.id=cc86b64693b6417e90ad9361460574b0 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:59.221710 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b9328-9fb9-4c01-bb65-9847497ab162 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:59.245343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:59.247593 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 389/762] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:26:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:26:59.283235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171f7a0-860f-4a07-aebd-dde2dc399e38 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:59.367914 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b9328-9fb9-4c01-bb65-9847497ab162 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:59.421441 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e0726-c0fd-49bc-98fb-b7fd55b8295a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:59.422383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e0726-c0fd-49bc-98fb-b7fd55b8295a None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:59.423506 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e0726-c0fd-49bc-98fb-b7fd55b8295a None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:59.423993 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e0726-c0fd-49bc-98fb-b7fd55b8295a None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:59.425612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e0726-c0fd-49bc-98fb-b7fd55b8295a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:59.426103 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e0726-c0fd-49bc-98fb-b7fd55b8295a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:59.428543 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e0726-c0fd-49bc-98fb-b7fd55b8295a None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:59.446536 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:59.455820 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171f7a0-860f-4a07-aebd-dde2dc399e38 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:59.459293 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 374/763] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Aug 4 04:26:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:59.465443 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e0726-c0fd-49bc-98fb-b7fd55b8295a None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:59.531768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5097f-f913-4c70-b21c-b5d02dbeea89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:59.534596 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5097f-f913-4c70-b21c-b5d02dbeea89 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:59.537742 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5097f-f913-4c70-b21c-b5d02dbeea89 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:59.538256 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5097f-f913-4c70-b21c-b5d02dbeea89 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:59.539108 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5097f-f913-4c70-b21c-b5d02dbeea89 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:59.540727 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5097f-f913-4c70-b21c-b5d02dbeea89 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:59.564460 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-5cd5097f-f913-4c70-b21c-b5d02dbeea89 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 04:26:59.564460 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:59.564460 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:26:59.564460 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:26:59.564460 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:26:59.564460 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:26:59.564460 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:26:59.564460 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:26:59.564460 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 04:26:59.564460 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:59.564460 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:26:59.564460 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:59.564460 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:59.564460 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:59.564460 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:59.564460 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:59.565466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:59.565466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:59.565466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:59.565466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:59.565466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:59.565466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:59.565466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:59.565466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:26:59.565466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:26:59.565466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:26:59.565466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:26:59.565466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:59.565466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:59.565466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:59.565466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:59.565466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:59.566344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:59.566344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:59.566344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:59.566344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:59.566344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:59.566344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:59.566344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:59.566344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:59.566344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:59.566344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:59.566344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:59.566344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:26:59.566344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:26:59.566344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 04:26:59.566344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:26:59.566344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 390/764] 10.222.0.47 () {56 vars in 1129 bytes} [Thu Aug 4 04:26:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:26:59.567786 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5097f-f913-4c70-b21c-b5d02dbeea89 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:59.593449 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:59.637581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e0726-c0fd-49bc-98fb-b7fd55b8295a None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:59.700627 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b7531-43a0-43aa-8c59-1d2a0a2030f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:59.701269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b7531-43a0-43aa-8c59-1d2a0a2030f6 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:59.701567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b7531-43a0-43aa-8c59-1d2a0a2030f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:59.701842 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b7531-43a0-43aa-8c59-1d2a0a2030f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:59.723251 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b7531-43a0-43aa-8c59-1d2a0a2030f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:59.724641 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b7531-43a0-43aa-8c59-1d2a0a2030f6 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:59.741167 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5097f-f913-4c70-b21c-b5d02dbeea89 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:59.754749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-ed2b7531-43a0-43aa-8c59-1d2a0a2030f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:59.754749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:26:59.754749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:26:59.754749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:26:59.754749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:26:59.754749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:26:59.754749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:26:59.754749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:26:59.754749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:26:59.754749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:26:59.754749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:26:59.754749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:26:59.754749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:26:59.754749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:26:59.754749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:26:59.755702 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:26:59.755702 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:26:59.755702 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:26:59.755702 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:26:59.755702 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:26:59.755702 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:26:59.755702 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:26:59.755702 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:26:59.755702 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:26:59.755702 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:26:59.755702 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:26:59.755702 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:26:59.755702 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:26:59.755702 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:26:59.755702 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:26:59.755702 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:26:59.755702 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:26:59.756596 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:26:59.756596 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:26:59.756596 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:26:59.756596 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:26:59.756596 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 375/765] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:26:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:26:59.765183 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b7531-43a0-43aa-8c59-1d2a0a2030f6 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:59.823135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb3b4e-8a0a-4f7c-9567-a204de11147d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:59.824136 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb3b4e-8a0a-4f7c-9567-a204de11147d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:59.824621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb3b4e-8a0a-4f7c-9567-a204de11147d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:59.825097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb3b4e-8a0a-4f7c-9567-a204de11147d None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:59.826017 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb3b4e-8a0a-4f7c-9567-a204de11147d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:59.827133 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb3b4e-8a0a-4f7c-9567-a204de11147d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:59.829589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb3b4e-8a0a-4f7c-9567-a204de11147d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:26:59.838883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 391/766] 10.222.0.47 () {56 vars in 1124 bytes} [Thu Aug 4 04:26:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:26:59.845504 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb3b4e-8a0a-4f7c-9567-a204de11147d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:26:59.909184 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:59.922502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b7531-43a0-43aa-8c59-1d2a0a2030f6 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:26:59.978421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:26:59.993747 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66917fa-140e-4a13-a1d5-4580ddb863c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:26:59.994462 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66917fa-140e-4a13-a1d5-4580ddb863c1 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:26:59.994748 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66917fa-140e-4a13-a1d5-4580ddb863c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:26:59.995022 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66917fa-140e-4a13-a1d5-4580ddb863c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:26:59.995720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66917fa-140e-4a13-a1d5-4580ddb863c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:26:59.996028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66917fa-140e-4a13-a1d5-4580ddb863c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:26:59.997393 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66917fa-140e-4a13-a1d5-4580ddb863c1 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:00.021348 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 376/767] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:00.026782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66917fa-140e-4a13-a1d5-4580ddb863c1 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:00.096178 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb3b4e-8a0a-4f7c-9567-a204de11147d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:00.149656 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66917fa-140e-4a13-a1d5-4580ddb863c1 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:00.178923 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9341c51-e9ee-4f49-ae99-57769ec31826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:00.180177 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9341c51-e9ee-4f49-ae99-57769ec31826 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:00.180611 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9341c51-e9ee-4f49-ae99-57769ec31826 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:00.181070 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9341c51-e9ee-4f49-ae99-57769ec31826 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:00.181973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9341c51-e9ee-4f49-ae99-57769ec31826 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:00.183429 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9341c51-e9ee-4f49-ae99-57769ec31826 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:00.185067 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9341c51-e9ee-4f49-ae99-57769ec31826 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:00.196464 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 392/768] 10.222.0.47 () {56 vars in 1124 bytes} [Thu Aug 4 04:26:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:00.203488 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9341c51-e9ee-4f49-ae99-57769ec31826 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:00.218252 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee6578-582e-42f4-967b-965c52a13123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:00.219449 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee6578-582e-42f4-967b-965c52a13123 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:00.220043 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee6578-582e-42f4-967b-965c52a13123 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:00.220534 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee6578-582e-42f4-967b-965c52a13123 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:00.236011 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee6578-582e-42f4-967b-965c52a13123 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:00.240173 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee6578-582e-42f4-967b-965c52a13123 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:00.263848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-a5ee6578-582e-42f4-967b-965c52a13123 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:00.263848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:00.263848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:00.263848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:00.263848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:00.263848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:00.263848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:00.263848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:00.263848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:00.263848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:00.263848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:00.263848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:00.263848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:27:00.263848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:27:00.263848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:27:00.264971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:27:00.264971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:00.264971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:00.264971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:00.264971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:00.264971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:00.264971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:00.264971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:00.264971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:00.264971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:00.264971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:00.264971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:00.264971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:00.264971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:00.264971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:00.264971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:00.264971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:00.266112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:27:00.266112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:27:00.266112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:27:00.266112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:27:00.266112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:27:00.266112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:27:00.266112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:00.266112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:00.276290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 377/769] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:27:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:00.283423 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee6578-582e-42f4-967b-965c52a13123 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:00.337815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:00.350745 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9341c51-e9ee-4f49-ae99-57769ec31826 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:00.385978 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b855ba-5249-48e5-97f3-88a9a4546bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:00.386888 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b855ba-5249-48e5-97f3-88a9a4546bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:00.387316 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b855ba-5249-48e5-97f3-88a9a4546bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:00.387736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b855ba-5249-48e5-97f3-88a9a4546bc7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:00.388663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b855ba-5249-48e5-97f3-88a9a4546bc7 None admin] RBAC: Adding query filter params (interface=public, service_id=6f4d37eada87462f8f2d85c3bbedf5e8, region_id=RegionOne) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:00.389121 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b855ba-5249-48e5-97f3-88a9a4546bc7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:00.390651 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b855ba-5249-48e5-97f3-88a9a4546bc7 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:00.399290 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 393/770] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Aug 4 04:27:00 2022] GET /identity/v3/endpoints?service_id=6f4d37eada87462f8f2d85c3bbedf5e8&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:00.403118 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:00.404552 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b855ba-5249-48e5-97f3-88a9a4546bc7 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:00.465225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee6578-582e-42f4-967b-965c52a13123 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:00.502853 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a689ff1-c99a-4547-92c3-8ea6125de85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:00.503345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a689ff1-c99a-4547-92c3-8ea6125de85b None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:00.503692 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a689ff1-c99a-4547-92c3-8ea6125de85b None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:00.503962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a689ff1-c99a-4547-92c3-8ea6125de85b None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:00.504767 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a689ff1-c99a-4547-92c3-8ea6125de85b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:00.505071 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a689ff1-c99a-4547-92c3-8ea6125de85b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:00.506755 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a689ff1-c99a-4547-92c3-8ea6125de85b None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:00.528889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 378/771] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:27:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:00.528889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b226d-910c-49b9-b4a2-84e727a366a0 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:00.528889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b226d-910c-49b9-b4a2-84e727a366a0 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:00.528889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b226d-910c-49b9-b4a2-84e727a366a0 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:00.531983 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 379/772] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:00.533220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b226d-910c-49b9-b4a2-84e727a366a0 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:00.546942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b855ba-5249-48e5-97f3-88a9a4546bc7 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:00.593281 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc621e6d-ced2-49a0-a584-0fadec393d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:00.594175 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc621e6d-ced2-49a0-a584-0fadec393d2d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:00.594608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc621e6d-ced2-49a0-a584-0fadec393d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:00.596148 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc621e6d-ced2-49a0-a584-0fadec393d2d None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:00.597351 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc621e6d-ced2-49a0-a584-0fadec393d2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:00.599398 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc621e6d-ced2-49a0-a584-0fadec393d2d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:00.607062 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 394/773] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Aug 4 04:27:00 2022] GET /identity/v3/services => generated 772 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:00.608803 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b49f4-34b3-4fe0-aa69-f43102244de2 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:00.609304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b49f4-34b3-4fe0-aa69-f43102244de2 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:00.610932 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b49f4-34b3-4fe0-aa69-f43102244de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:00.636724 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-150b49f4-34b3-4fe0-aa69-f43102244de2 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:00.639182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-150b49f4-34b3-4fe0-aa69-f43102244de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:00.653176 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b226d-910c-49b9-b4a2-84e727a366a0 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:00.656758 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:00.667030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:00.718608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6570110-bbeb-491b-a2e5-b3708fe5f21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:00.726037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6570110-bbeb-491b-a2e5-b3708fe5f21d None admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:00.726810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6570110-bbeb-491b-a2e5-b3708fe5f21d None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:00.727317 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6570110-bbeb-491b-a2e5-b3708fe5f21d None admin] PATH_INFO: `/v3/projects/f37a5f3140ec431da8ea14568654114a/users/caa2901451aa4554bdfcac1bddbc620a/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:00.730098 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6570110-bbeb-491b-a2e5-b3708fe5f21d None admin] RBAC: Authorizing `identity:create_grant(project_id=f37a5f3140ec431da8ea14568654114a, user_id=caa2901451aa4554bdfcac1bddbc620a, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:00.732171 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6570110-bbeb-491b-a2e5-b3708fe5f21d None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:00.759972 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 380/774] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Aug 4 04:27:00 2022] PUT /identity/v3/projects/f37a5f3140ec431da8ea14568654114a/users/caa2901451aa4554bdfcac1bddbc620a/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:27:00.787582 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 395/775] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:00.832918 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6570110-bbeb-491b-a2e5-b3708fe5f21d None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:00.952053 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6570110-bbeb-491b-a2e5-b3708fe5f21d None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:00.991297 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9988e-1c72-475f-bab9-b499de06d503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:00.991984 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9988e-1c72-475f-bab9-b499de06d503 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:00.992260 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9988e-1c72-475f-bab9-b499de06d503 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:00.992580 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9988e-1c72-475f-bab9-b499de06d503 None admin] PATH_INFO: `/v3/services/network` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:00.993213 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9988e-1c72-475f-bab9-b499de06d503 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:00.994598 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9988e-1c72-475f-bab9-b499de06d503 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:01.006263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-50b9988e-1c72-475f-bab9-b499de06d503 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 04:27:01.006263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:01.006263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:27:01.006263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:27:01.006263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:27:01.006263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:27:01.006263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:27:01.006263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:01.006263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 04:27:01.006263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:01.006263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:27:01.006263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:01.006263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:01.006263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:01.006263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:01.006263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:01.006263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:01.007265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:01.007265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:01.007265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:01.007265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:01.007265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:01.007265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:01.007265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:27:01.007265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:27:01.007265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:27:01.007265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:27:01.007265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:01.007265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:01.007265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:01.007265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:01.007265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:01.007265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:01.008067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:01.008067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:01.008067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:01.008067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:01.008067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:01.008067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:01.008067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:01.008067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:01.008067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:01.008067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:01.008067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:27:01.008067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:01.008067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 04:27:01.008067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:01.008067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 381/776] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:27:00 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:01.013387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b49f4-34b3-4fe0-aa69-f43102244de2 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:01.156303 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b49f4-34b3-4fe0-aa69-f43102244de2 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:01.208822 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe26aad-e278-4194-aa75-fe5fe3306613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:01.209447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe26aad-e278-4194-aa75-fe5fe3306613 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:01.209746 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe26aad-e278-4194-aa75-fe5fe3306613 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:01.210023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe26aad-e278-4194-aa75-fe5fe3306613 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:01.210721 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe26aad-e278-4194-aa75-fe5fe3306613 None admin] RBAC: Adding query filter params (name=network) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:01.211028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe26aad-e278-4194-aa75-fe5fe3306613 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:01.212399 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe26aad-e278-4194-aa75-fe5fe3306613 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:01.222204 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 396/777] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Aug 4 04:27:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:01.227373 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9988e-1c72-475f-bab9-b499de06d503 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:01.240911 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075f373-d740-438d-9fe6-5c316b60b753 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:01.241269 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075f373-d740-438d-9fe6-5c316b60b753 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:01.241580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075f373-d740-438d-9fe6-5c316b60b753 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:01.243309 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 397/778] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:27:01.248271 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef5880-f99d-4c6f-bb0b-a1373420dc48 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:01.248556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef5880-f99d-4c6f-bb0b-a1373420dc48 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:01.248835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef5880-f99d-4c6f-bb0b-a1373420dc48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:01.279853 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-8fef5880-f99d-4c6f-bb0b-a1373420dc48 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:01.282852 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8fef5880-f99d-4c6f-bb0b-a1373420dc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:01.292253 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:01.294176 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:01.421640 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 398/779] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 04:27:01.435343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9988e-1c72-475f-bab9-b499de06d503 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:01.461633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef5880-f99d-4c6f-bb0b-a1373420dc48 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:01.492405 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a443b11-7274-4ace-9dda-c55060359631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:01.493474 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a443b11-7274-4ace-9dda-c55060359631 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:01.493981 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a443b11-7274-4ace-9dda-c55060359631 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:01.494446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a443b11-7274-4ace-9dda-c55060359631 None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:01.495325 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a443b11-7274-4ace-9dda-c55060359631 None admin] RBAC: Adding query filter params (type=network) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:01.495910 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a443b11-7274-4ace-9dda-c55060359631 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:01.497450 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a443b11-7274-4ace-9dda-c55060359631 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:01.507455 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 382/780] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Aug 4 04:27:01 2022] GET /identity/v3/services?type=network => generated 350 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:01.514444 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a443b11-7274-4ace-9dda-c55060359631 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:01.632075 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:01.637839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef5880-f99d-4c6f-bb0b-a1373420dc48 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:01.639300 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a443b11-7274-4ace-9dda-c55060359631 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:01.681702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fab19e-1c7b-4bd9-b008-069588b17f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:01.681702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fab19e-1c7b-4bd9-b008-069588b17f31 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:01.681702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fab19e-1c7b-4bd9-b008-069588b17f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:01.681702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fab19e-1c7b-4bd9-b008-069588b17f31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:01.686282 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf30663-f3ac-4d26-9c6e-c5e118db3c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:01.687077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf30663-f3ac-4d26-9c6e-c5e118db3c50 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:01.689745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf30663-f3ac-4d26-9c6e-c5e118db3c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:01.690181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf30663-f3ac-4d26-9c6e-c5e118db3c50 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:01.691125 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf30663-f3ac-4d26-9c6e-c5e118db3c50 None admin] RBAC: Adding query filter params (interface=public, service_id=c3e7fa3c4f7c48e6aa4cf4d4fac41c96, region_id=RegionOne) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:01.691929 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf30663-f3ac-4d26-9c6e-c5e118db3c50 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:01.694207 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf30663-f3ac-4d26-9c6e-c5e118db3c50 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:01.701885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:01.705180 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fab19e-1c7b-4bd9-b008-069588b17f31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:27:01.710346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-95fab19e-1c7b-4bd9-b008-069588b17f31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:27:01.710346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:01.710346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:01.710346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:01.710346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:01.710346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:01.710346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:01.710346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:01.710346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:01.710346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:01.710346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:01.710346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:01.710346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:27:01.710346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:27:01.710346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:27:01.710346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:27:01.711483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:27:01.711483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:27:01.711483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:27:01.711483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:27:01.711483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:01.711483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:01.711483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:01.711483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:01.711483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:01.711483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:01.711483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:01.711483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:01.711483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:01.711483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:01.711483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:01.711483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:01.712419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:01.712419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:01.712419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:01.712419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:01.712419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:27:01.712419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:27:01.712419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:27:01.712419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:27:01.712419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:27:01.712419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:27:01.712419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:27:01.712419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:01.713213 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 399/781] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:27:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:01.718024 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fab19e-1c7b-4bd9-b008-069588b17f31 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:01.718482 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 383/782] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Aug 4 04:27:01 2022] GET /identity/v3/endpoints?service_id=c3e7fa3c4f7c48e6aa4cf4d4fac41c96&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:01.724062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf30663-f3ac-4d26-9c6e-c5e118db3c50 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:01.852388 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fab19e-1c7b-4bd9-b008-069588b17f31 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:01.857963 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf30663-f3ac-4d26-9c6e-c5e118db3c50 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:01.888831 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b917460-44b7-4c5c-9876-c266b88c98e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:01.889369 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b917460-44b7-4c5c-9876-c266b88c98e2 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:01.889680 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b917460-44b7-4c5c-9876-c266b88c98e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:01.889961 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b917460-44b7-4c5c-9876-c266b88c98e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:01.890901 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b917460-44b7-4c5c-9876-c266b88c98e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:01.891210 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b917460-44b7-4c5c-9876-c266b88c98e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:01.893123 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b917460-44b7-4c5c-9876-c266b88c98e2 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:01.900230 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a01b48-bbff-4670-8c72-1df3141e3501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:01.900954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a01b48-bbff-4670-8c72-1df3141e3501 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:01.901236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a01b48-bbff-4670-8c72-1df3141e3501 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:01.901507 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a01b48-bbff-4670-8c72-1df3141e3501 None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:01.902183 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a01b48-bbff-4670-8c72-1df3141e3501 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:01.902727 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 400/783] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:27:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:01.903528 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a01b48-bbff-4670-8c72-1df3141e3501 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:01.904038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4e630-4f3d-465f-b8ef-c32a17f6afb6 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:01.904517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4e630-4f3d-465f-b8ef-c32a17f6afb6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:01.904866 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4e630-4f3d-465f-b8ef-c32a17f6afb6 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:01.910446 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 401/784] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:01.911348 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4e630-4f3d-465f-b8ef-c32a17f6afb6 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:01.914417 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 384/785] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Aug 4 04:27:01 2022] GET /identity/v3/services => generated 772 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:01.916765 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0f29c-9130-4215-8fe0-37ef4e349240 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:01.917177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0f29c-9130-4215-8fe0-37ef4e349240 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:01.917564 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0f29c-9130-4215-8fe0-37ef4e349240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:01.948879 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-e0d0f29c-9130-4215-8fe0-37ef4e349240 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:01.949079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-e0d0f29c-9130-4215-8fe0-37ef4e349240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:01.992882 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:02.019116 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:02.071019 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 385/786] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:02.088439 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d0f29c-9130-4215-8fe0-37ef4e349240 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:02.109765 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4e630-4f3d-465f-b8ef-c32a17f6afb6 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:02.156445 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c48a23-bd92-4b9e-80d5-0139ed53cd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:02.157020 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c48a23-bd92-4b9e-80d5-0139ed53cd50 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:02.158066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c48a23-bd92-4b9e-80d5-0139ed53cd50 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:02.158348 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c48a23-bd92-4b9e-80d5-0139ed53cd50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:02.170340 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c48a23-bd92-4b9e-80d5-0139ed53cd50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:02.171802 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c48a23-bd92-4b9e-80d5-0139ed53cd50 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:02.189321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-b0c48a23-bd92-4b9e-80d5-0139ed53cd50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:02.189321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:02.189321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:02.189321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:02.189321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:02.189321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:02.189321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:02.189321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:02.189321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:02.189321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:02.189321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:02.189321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:02.189321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:27:02.189321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:27:02.189321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:27:02.190276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:27:02.190276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:02.190276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:02.190276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:02.190276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:02.190276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:02.190276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:02.190276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:02.190276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:02.190276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:02.190276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:02.190276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:02.190276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:02.190276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:02.190276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:02.190276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:02.190276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:02.191125 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:27:02.191125 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:27:02.191125 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:02.191125 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:02.191125 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 402/787] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:02.196417 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c48a23-bd92-4b9e-80d5-0139ed53cd50 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:02.227468 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d0f29c-9130-4215-8fe0-37ef4e349240 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:02.273485 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fddb92b-59e9-4dab-9b71-85cd027699e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:02.274204 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddb92b-59e9-4dab-9b71-85cd027699e1 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:02.274490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddb92b-59e9-4dab-9b71-85cd027699e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:02.275015 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddb92b-59e9-4dab-9b71-85cd027699e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:02.289519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fddb92b-59e9-4dab-9b71-85cd027699e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:27:02.296921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-7fddb92b-59e9-4dab-9b71-85cd027699e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:27:02.296921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:02.296921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:02.296921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:02.296921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:02.296921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:02.296921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:02.296921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:02.296921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:02.296921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:02.296921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:02.296921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:02.296921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:27:02.296921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:27:02.296921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:27:02.296921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:27:02.298008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:27:02.298008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:27:02.298008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:27:02.298008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:27:02.298008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:02.298008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:02.298008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:02.298008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:02.298008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:02.298008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:02.298008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:02.298008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:02.298008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:02.298008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:02.298008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:02.298008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:02.298828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:02.298828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:02.298828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:02.298828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:02.298828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:27:02.298828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:27:02.298828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:27:02.298828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:27:02.298828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:27:02.298828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:27:02.298828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:27:02.298828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:02.298828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 386/788] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Aug 4 04:27:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:02.306194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fddb92b-59e9-4dab-9b71-85cd027699e1 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:02.317880 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c48a23-bd92-4b9e-80d5-0139ed53cd50 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:02.332651 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:02.337901 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:02.355187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55e57b-324e-4cb3-922b-015748175450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:02.355860 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55e57b-324e-4cb3-922b-015748175450 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:02.356143 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55e57b-324e-4cb3-922b-015748175450 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:02.356419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55e57b-324e-4cb3-922b-015748175450 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:02.360092 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e55e57b-324e-4cb3-922b-015748175450 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:02.360092 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e55e57b-324e-4cb3-922b-015748175450 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:02.360092 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e55e57b-324e-4cb3-922b-015748175450 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:02.372184 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 403/789] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:02.377579 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55e57b-324e-4cb3-922b-015748175450 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:02.515215 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fddb92b-59e9-4dab-9b71-85cd027699e1 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:02.577693 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55e57b-324e-4cb3-922b-015748175450 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:02.599167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687be42-e586-4b7d-bcf5-e66f5af2f033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:02.599905 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687be42-e586-4b7d-bcf5-e66f5af2f033 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:02.600190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687be42-e586-4b7d-bcf5-e66f5af2f033 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:02.600461 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687be42-e586-4b7d-bcf5-e66f5af2f033 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:02.601185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687be42-e586-4b7d-bcf5-e66f5af2f033 None admin] RBAC: Adding query filter params (name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:02.601489 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687be42-e586-4b7d-bcf5-e66f5af2f033 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:02.602988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687be42-e586-4b7d-bcf5-e66f5af2f033 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:02.622884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 387/790] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Aug 4 04:27:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:02.627543 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a8ed3e-7970-4acb-a404-753ebe659e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:02.628206 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8ed3e-7970-4acb-a404-753ebe659e3c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:02.628487 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8ed3e-7970-4acb-a404-753ebe659e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:02.628758 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8ed3e-7970-4acb-a404-753ebe659e3c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:02.634184 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687be42-e586-4b7d-bcf5-e66f5af2f033 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:02.650361 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a8ed3e-7970-4acb-a404-753ebe659e3c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:02.651831 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a8ed3e-7970-4acb-a404-753ebe659e3c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:02.675760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-45a8ed3e-7970-4acb-a404-753ebe659e3c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 04:27:02.675760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:02.675760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:02.675760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:02.675760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:02.675760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:02.675760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:02.675760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:02.675760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:02.675760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:02.675760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:02.675760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:02.675760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:27:02.675760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:27:02.675760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:27:02.675760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:27:02.676777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:02.676777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:02.676777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:27:02.676777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:02.676777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:27:02.676777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:02.676777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:02.676777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:02.676777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:02.676777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:02.676777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:02.676777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:02.676777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:02.676777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:02.676777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:02.676777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:02.676777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:02.676777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:02.677781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:02.677781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:02.677781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:27:02.677781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:27:02.677781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:27:02.677781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:27:02.677781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:27:02.677781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:27:02.677781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 04:27:02.677781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:02.684593 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 404/791] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:27:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:02.690769 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a8ed3e-7970-4acb-a404-753ebe659e3c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:02.737787 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:02.752810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:02.793018 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687be42-e586-4b7d-bcf5-e66f5af2f033 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:02.861506 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45581dfe-68eb-492c-b4c6-4b749f9d962c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:02.862190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45581dfe-68eb-492c-b4c6-4b749f9d962c None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:02.862704 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45581dfe-68eb-492c-b4c6-4b749f9d962c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:02.862979 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45581dfe-68eb-492c-b4c6-4b749f9d962c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:02.877868 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a8ed3e-7970-4acb-a404-753ebe659e3c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:02.887688 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45581dfe-68eb-492c-b4c6-4b749f9d962c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:02.889226 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45581dfe-68eb-492c-b4c6-4b749f9d962c None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:02.914135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-45581dfe-68eb-492c-b4c6-4b749f9d962c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:02.914135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:02.914135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:02.914135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:02.914135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:02.914135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:02.914135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:02.914135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:02.914135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:02.914135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:02.914135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:02.914135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:02.914135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:27:02.914135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:27:02.914135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:27:02.916283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:27:02.916283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:02.916283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:02.916283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:02.916283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:02.916283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:02.916283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:02.916283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:02.916283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:02.916283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:02.916283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:02.916283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:02.916283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:02.916283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:02.916283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:02.916283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:02.916283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:02.918484 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:27:02.918484 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:27:02.918484 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:02.918484 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:02.918484 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 388/792] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:02.924796 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45581dfe-68eb-492c-b4c6-4b749f9d962c None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:02.960937 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8212c9-61e7-44d3-b68d-eb23ec51debf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:02.961578 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8212c9-61e7-44d3-b68d-eb23ec51debf None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:02.961867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8212c9-61e7-44d3-b68d-eb23ec51debf None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:02.962144 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8212c9-61e7-44d3-b68d-eb23ec51debf None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:02.963006 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8212c9-61e7-44d3-b68d-eb23ec51debf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:02.963320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8212c9-61e7-44d3-b68d-eb23ec51debf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:02.965323 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8212c9-61e7-44d3-b68d-eb23ec51debf None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:02.988940 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 405/793] 10.222.0.47 () {56 vars in 1135 bytes} [Thu Aug 4 04:27:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:02.994393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8212c9-61e7-44d3-b68d-eb23ec51debf None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:03.102238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:03.114668 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8212c9-61e7-44d3-b68d-eb23ec51debf None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:03.154000 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc6801-d942-4e27-a2d5-1d2d3cff9f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:03.154651 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc6801-d942-4e27-a2d5-1d2d3cff9f27 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:03.155176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc6801-d942-4e27-a2d5-1d2d3cff9f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:03.155462 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc6801-d942-4e27-a2d5-1d2d3cff9f27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:03.168328 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc6801-d942-4e27-a2d5-1d2d3cff9f27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:03.169820 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc6801-d942-4e27-a2d5-1d2d3cff9f27 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:03.177524 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45581dfe-68eb-492c-b4c6-4b749f9d962c None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:03.186387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-fcdc6801-d942-4e27-a2d5-1d2d3cff9f27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:03.186387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:03.186387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:03.186387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:03.186387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:03.186387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:03.186387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:03.186387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:03.186387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:03.186387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:03.186387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:03.186387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:03.186387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:27:03.186387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:27:03.186387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:27:03.188520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:27:03.188520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:03.188520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:03.188520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:03.188520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:03.188520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:03.188520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:03.188520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:03.188520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:03.188520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:03.188520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:03.188520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:03.188520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:03.188520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:03.188520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:03.188520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:03.188520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:03.190022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:27:03.190022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:27:03.190022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:03.190022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:03.190022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 406/794] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:03.193439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc6801-d942-4e27-a2d5-1d2d3cff9f27 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:03.228371 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:03.239529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ab650-4a2f-4f63-b437-2e3144189dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:03.240393 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ab650-4a2f-4f63-b437-2e3144189dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:03.240823 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ab650-4a2f-4f63-b437-2e3144189dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:03.241233 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ab650-4a2f-4f63-b437-2e3144189dd7 None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:03.242092 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ab650-4a2f-4f63-b437-2e3144189dd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:03.242632 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ab650-4a2f-4f63-b437-2e3144189dd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:03.244183 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ab650-4a2f-4f63-b437-2e3144189dd7 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:03.258596 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 389/795] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:03.260146 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166972d1-a5e0-4dfe-810e-b0657509f818 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:03.260628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166972d1-a5e0-4dfe-810e-b0657509f818 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:03.261171 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166972d1-a5e0-4dfe-810e-b0657509f818 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:03.267180 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 390/796] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:03.268391 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166972d1-a5e0-4dfe-810e-b0657509f818 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:03.315767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc6801-d942-4e27-a2d5-1d2d3cff9f27 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:03.352052 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0354ac-faf0-49cf-b8cf-cd42b8658ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:03.352744 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0354ac-faf0-49cf-b8cf-cd42b8658ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:03.353082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0354ac-faf0-49cf-b8cf-cd42b8658ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:03.353421 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0354ac-faf0-49cf-b8cf-cd42b8658ea3 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:03.354215 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0354ac-faf0-49cf-b8cf-cd42b8658ea3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:03.354540 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0354ac-faf0-49cf-b8cf-cd42b8658ea3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:03.356200 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0354ac-faf0-49cf-b8cf-cd42b8658ea3 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:03.370085 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 407/797] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:03.372073 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad034619-614b-4e70-8eb9-053d58bd1751 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:03.372617 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad034619-614b-4e70-8eb9-053d58bd1751 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:03.373096 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad034619-614b-4e70-8eb9-053d58bd1751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:03.383864 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:03.388398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166972d1-a5e0-4dfe-810e-b0657509f818 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:03.399478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ad034619-614b-4e70-8eb9-053d58bd1751 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:03.401207 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ad034619-614b-4e70-8eb9-053d58bd1751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:03.422431 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635feee6-33be-4eae-a485-ec9f949939e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:03.423626 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635feee6-33be-4eae-a485-ec9f949939e5 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:03.423883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635feee6-33be-4eae-a485-ec9f949939e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:03.424346 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635feee6-33be-4eae-a485-ec9f949939e5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:03.431590 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635feee6-33be-4eae-a485-ec9f949939e5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:03.433004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635feee6-33be-4eae-a485-ec9f949939e5 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:03.445058 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-635feee6-33be-4eae-a485-ec9f949939e5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 04:27:03.445058 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:03.445058 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:03.445058 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:03.445058 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:03.445058 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:03.445058 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:03.445058 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:03.445058 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:03.445058 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:03.445058 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:03.445058 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:03.445058 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:27:03.445058 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:27:03.445058 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:27:03.445058 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:27:03.446105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:03.446105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:03.446105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:27:03.446105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:03.446105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:27:03.446105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:03.446105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:03.446105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:03.446105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:03.446105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:03.446105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:03.446105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:03.446105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:03.446105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:03.446105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:03.446105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:03.446105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:03.446105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:03.447070 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:03.447070 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:03.447070 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:27:03.447070 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:27:03.447070 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:27:03.447070 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:27:03.447070 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:27:03.447070 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:27:03.447070 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 04:27:03.447070 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:03.447070 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 391/798] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Aug 4 04:27:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:03.447644 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef1868-3738-4dd0-8c12-d5c4215c33b5 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:03.447906 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef1868-3738-4dd0-8c12-d5c4215c33b5 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:03.448175 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef1868-3738-4dd0-8c12-d5c4215c33b5 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:03.453496 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 392/799] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:03.453496 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef1868-3738-4dd0-8c12-d5c4215c33b5 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:03.488756 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 408/800] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:03.490468 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad034619-614b-4e70-8eb9-053d58bd1751 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:03.514938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:03.575191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef1868-3738-4dd0-8c12-d5c4215c33b5 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:03.602563 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad034619-614b-4e70-8eb9-053d58bd1751 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:03.612759 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38d472-4068-4cc3-bd5a-13f37c91d262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:03.613333 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38d472-4068-4cc3-bd5a-13f37c91d262 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:03.613637 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38d472-4068-4cc3-bd5a-13f37c91d262 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:03.614038 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38d472-4068-4cc3-bd5a-13f37c91d262 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:03.620810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38d472-4068-4cc3-bd5a-13f37c91d262 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:03.622398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38d472-4068-4cc3-bd5a-13f37c91d262 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:03.634367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-4c38d472-4068-4cc3-bd5a-13f37c91d262 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:03.634367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:03.634367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:03.634367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:03.634367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:03.634367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:03.634367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:03.634367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:03.634367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:03.634367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:03.634367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:03.634367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:03.634367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:27:03.634367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:27:03.634367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:27:03.635311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:27:03.635311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:03.635311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:03.635311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:03.635311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:03.635311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:03.635311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:03.635311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:03.635311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:03.635311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:03.635311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:03.635311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:03.635311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:03.635311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:03.635311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:03.635311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:03.635311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:03.636177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:27:03.636177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:27:03.636177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:27:03.636177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:27:03.636177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:27:03.636177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:27:03.636177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:03.636177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:03.636177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 393/801] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:27:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:03.637254 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d9c35-d04f-4250-ab0a-a2ea37d5fc82 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:03.637254 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d9c35-d04f-4250-ab0a-a2ea37d5fc82 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:03.637254 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d9c35-d04f-4250-ab0a-a2ea37d5fc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:03.646805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b753d-4680-476d-ae15-590ffe00d773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:03.647423 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b753d-4680-476d-ae15-590ffe00d773 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:03.647698 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b753d-4680-476d-ae15-590ffe00d773 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:03.647991 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b753d-4680-476d-ae15-590ffe00d773 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:03.648786 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b753d-4680-476d-ae15-590ffe00d773 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:03.649087 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b753d-4680-476d-ae15-590ffe00d773 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:03.650539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b753d-4680-476d-ae15-590ffe00d773 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:03.652292 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:03.657963 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-577d9c35-d04f-4250-ab0a-a2ea37d5fc82 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:03.659146 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-577d9c35-d04f-4250-ab0a-a2ea37d5fc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:03.676087 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 409/802] 10.222.0.47 () {56 vars in 1149 bytes} [Thu Aug 4 04:27:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:03.676087 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b753d-4680-476d-ae15-590ffe00d773 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:03.736966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 394/803] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:03.737715 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d9c35-d04f-4250-ab0a-a2ea37d5fc82 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:03.785980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b753d-4680-476d-ae15-590ffe00d773 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:03.788151 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:03.821919 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27599f-2579-412e-b976-b44cf39b7862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:03.822567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27599f-2579-412e-b976-b44cf39b7862 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:03.822848 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27599f-2579-412e-b976-b44cf39b7862 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:03.823122 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27599f-2579-412e-b976-b44cf39b7862 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:03.823749 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27599f-2579-412e-b976-b44cf39b7862 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:03.825117 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27599f-2579-412e-b976-b44cf39b7862 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:03.834194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-db27599f-2579-412e-b976-b44cf39b7862 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 04:27:03.834194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:03.834194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:27:03.834194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:27:03.834194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:27:03.834194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:27:03.834194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:27:03.834194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:03.834194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 04:27:03.834194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:03.834194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:27:03.834194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:03.834194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:03.834194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:03.834194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:03.834194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:03.835147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:03.835147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:03.835147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:03.835147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:03.835147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:03.835147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:03.835147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:03.835147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:27:03.835147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:27:03.835147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:27:03.835147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:27:03.835147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:03.835147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:03.835147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:03.835147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:03.835147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:03.835979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:03.835979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:03.835979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:03.835979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:03.835979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:03.835979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:03.835979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:03.835979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:03.835979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:03.835979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:03.835979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:03.835979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:27:03.835979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:03.835979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 04:27:03.835979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:03.835979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 410/804] 10.222.0.47 () {56 vars in 1129 bytes} [Thu Aug 4 04:27:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:03.836816 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27599f-2579-412e-b976-b44cf39b7862 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:03.859361 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d9c35-d04f-4250-ab0a-a2ea37d5fc82 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:03.899931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28109335-ff74-457f-bb30-8624d814d247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:03.900471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28109335-ff74-457f-bb30-8624d814d247 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:03.900748 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28109335-ff74-457f-bb30-8624d814d247 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:03.901016 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28109335-ff74-457f-bb30-8624d814d247 None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:03.901787 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28109335-ff74-457f-bb30-8624d814d247 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:03.902087 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28109335-ff74-457f-bb30-8624d814d247 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:03.903288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:03.903488 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28109335-ff74-457f-bb30-8624d814d247 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:03.921055 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 395/805] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:27:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:03.921055 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28109335-ff74-457f-bb30-8624d814d247 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:03.959576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27599f-2579-412e-b976-b44cf39b7862 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:03.995037 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4121a5c-5670-4a55-8c0b-3c948f0b7acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:03.995683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4121a5c-5670-4a55-8c0b-3c948f0b7acc None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:03.995963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4121a5c-5670-4a55-8c0b-3c948f0b7acc None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:03.996237 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4121a5c-5670-4a55-8c0b-3c948f0b7acc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:04.007036 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4121a5c-5670-4a55-8c0b-3c948f0b7acc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:04.008485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4121a5c-5670-4a55-8c0b-3c948f0b7acc None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:04.020178 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:04.023861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-c4121a5c-5670-4a55-8c0b-3c948f0b7acc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:04.023861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:04.023861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:04.023861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:04.023861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:04.023861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:04.023861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:04.023861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:04.023861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:04.023861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:04.023861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:04.023861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:04.023861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:27:04.023861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:27:04.023861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:27:04.024726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:27:04.024726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:04.024726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:04.024726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:04.024726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:04.024726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:04.024726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:04.024726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:04.024726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:04.024726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:04.024726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:04.024726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:04.024726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:04.024726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:04.024726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:04.024726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:04.024726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:04.025957 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:27:04.025957 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:27:04.025957 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:04.025957 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:04.026172 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28109335-ff74-457f-bb30-8624d814d247 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:04.029888 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 411/806] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:04.030723 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4121a5c-5670-4a55-8c0b-3c948f0b7acc None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:04.062022 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb4312-554d-477b-aed0-3c13837fc811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:04.062713 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb4312-554d-477b-aed0-3c13837fc811 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:04.063039 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb4312-554d-477b-aed0-3c13837fc811 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:04.063462 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb4312-554d-477b-aed0-3c13837fc811 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:04.074025 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb4312-554d-477b-aed0-3c13837fc811 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:27:04.080030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-d1fb4312-554d-477b-aed0-3c13837fc811 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:27:04.080030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:04.080030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:04.080030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:04.080030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:04.080030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:04.080030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:04.080030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:04.080030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:04.080030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:04.080030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:04.080030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:04.080030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:27:04.080030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:27:04.080030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:27:04.080030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:27:04.081051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:27:04.081051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:27:04.081051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:27:04.081051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:27:04.081051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:04.081051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:04.081051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:04.081051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:04.081051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:04.081051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:04.081051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:04.081051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:04.081051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:04.081051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:04.081051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:04.081051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:04.081897 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:04.081897 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:04.081897 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:04.081897 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:04.081897 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:27:04.081897 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:27:04.081897 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:27:04.081897 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:27:04.081897 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:27:04.081897 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:27:04.081897 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:27:04.081897 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:04.081897 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 396/807] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:27:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:04.081897 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb4312-554d-477b-aed0-3c13837fc811 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:04.142771 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4121a5c-5670-4a55-8c0b-3c948f0b7acc None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:04.151973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:04.179153 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7afbb70-eeaa-479c-a8e4-256ee5fdf50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:04.179768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7afbb70-eeaa-479c-a8e4-256ee5fdf50c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:04.180159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7afbb70-eeaa-479c-a8e4-256ee5fdf50c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:04.180438 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7afbb70-eeaa-479c-a8e4-256ee5fdf50c None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:04.181238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7afbb70-eeaa-479c-a8e4-256ee5fdf50c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:04.181553 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7afbb70-eeaa-479c-a8e4-256ee5fdf50c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:04.183050 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7afbb70-eeaa-479c-a8e4-256ee5fdf50c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:04.192753 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 412/808] 10.222.0.47 () {56 vars in 1124 bytes} [Thu Aug 4 04:27:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:04.192900 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb4312-554d-477b-aed0-3c13837fc811 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:04.193001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7afbb70-eeaa-479c-a8e4-256ee5fdf50c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:04.228996 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0041c-f636-4e31-9a86-722e6995a1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:04.229704 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0041c-f636-4e31-9a86-722e6995a1ed None admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:04.229982 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0041c-f636-4e31-9a86-722e6995a1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:04.230253 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0041c-f636-4e31-9a86-722e6995a1ed None admin] PATH_INFO: `/v3/projects/f37a5f3140ec431da8ea14568654114a/users/b1e539006166434790823b839bd745d5/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:04.232170 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a0041c-f636-4e31-9a86-722e6995a1ed None admin] RBAC: Authorizing `identity:create_grant(project_id=f37a5f3140ec431da8ea14568654114a, user_id=b1e539006166434790823b839bd745d5, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:04.233609 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a0041c-f636-4e31-9a86-722e6995a1ed None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:04.246036 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 397/809] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Aug 4 04:27:04 2022] PUT /identity/v3/projects/f37a5f3140ec431da8ea14568654114a/users/b1e539006166434790823b839bd745d5/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:27:04.246840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0041c-f636-4e31-9a86-722e6995a1ed None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:04.268869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:04.307623 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7afbb70-eeaa-479c-a8e4-256ee5fdf50c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:04.348228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1d3bce-1a3f-4bbb-92e3-30f856c5ec16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:04.348860 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d3bce-1a3f-4bbb-92e3-30f856c5ec16 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:04.349139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d3bce-1a3f-4bbb-92e3-30f856c5ec16 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:04.349412 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d3bce-1a3f-4bbb-92e3-30f856c5ec16 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:04.350146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1d3bce-1a3f-4bbb-92e3-30f856c5ec16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:04.350515 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1d3bce-1a3f-4bbb-92e3-30f856c5ec16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:04.351968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1d3bce-1a3f-4bbb-92e3-30f856c5ec16 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:04.367163 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 413/810] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:04.368052 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1d3bce-1a3f-4bbb-92e3-30f856c5ec16 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:04.369707 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0041c-f636-4e31-9a86-722e6995a1ed None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:04.410692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:04.416671 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246aaec-1a1f-4148-8b5d-4c3dbcf39a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:04.417181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246aaec-1a1f-4148-8b5d-4c3dbcf39a64 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:04.417463 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246aaec-1a1f-4148-8b5d-4c3dbcf39a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:04.417769 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246aaec-1a1f-4148-8b5d-4c3dbcf39a64 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:04.418500 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246aaec-1a1f-4148-8b5d-4c3dbcf39a64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:04.418811 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246aaec-1a1f-4148-8b5d-4c3dbcf39a64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:04.420438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246aaec-1a1f-4148-8b5d-4c3dbcf39a64 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:04.429753 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 398/811] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:27:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:04.431002 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c67a79-177b-418d-a950-fd355e170219 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:04.431287 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c67a79-177b-418d-a950-fd355e170219 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:04.431564 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c67a79-177b-418d-a950-fd355e170219 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:04.432898 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 399/812] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:04.433456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c67a79-177b-418d-a950-fd355e170219 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:04.497005 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1d3bce-1a3f-4bbb-92e3-30f856c5ec16 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:04.522461 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:04.540856 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9f317-c310-4ab9-84df-a334c1e5478b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:04.541504 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9f317-c310-4ab9-84df-a334c1e5478b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:04.541832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9f317-c310-4ab9-84df-a334c1e5478b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:04.542113 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9f317-c310-4ab9-84df-a334c1e5478b None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:04.542793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c9f317-c310-4ab9-84df-a334c1e5478b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:04.543097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c9f317-c310-4ab9-84df-a334c1e5478b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:04.544148 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c67a79-177b-418d-a950-fd355e170219 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:04.544463 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c9f317-c310-4ab9-84df-a334c1e5478b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:04.554298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 414/813] 10.222.0.47 () {56 vars in 1124 bytes} [Thu Aug 4 04:27:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:04.554298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9f317-c310-4ab9-84df-a334c1e5478b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:04.584401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56cc1e5-8233-46a1-9549-44dccad3d4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:04.585324 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cc1e5-8233-46a1-9549-44dccad3d4bb None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:04.585764 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cc1e5-8233-46a1-9549-44dccad3d4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:04.586186 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cc1e5-8233-46a1-9549-44dccad3d4bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:04.593148 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56cc1e5-8233-46a1-9549-44dccad3d4bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:04.595348 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56cc1e5-8233-46a1-9549-44dccad3d4bb None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:04.607095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-a56cc1e5-8233-46a1-9549-44dccad3d4bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:04.607095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:04.607095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:04.607095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:04.607095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:04.607095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:04.607095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:04.607095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:04.607095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:04.607095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:04.607095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:04.607095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:04.607095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:27:04.607095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:27:04.607095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:27:04.608163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:27:04.608163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:04.608163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:04.608163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:04.608163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:04.608163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:04.608163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:04.608163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:04.608163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:04.608163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:04.608163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:04.608163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:04.608163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:04.608163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:04.608163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:04.608163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:04.608163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:04.609244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:27:04.609244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:27:04.609244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:27:04.609244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:27:04.609244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:27:04.609244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:27:04.609244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:04.609244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:04.611108 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 400/814] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:27:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:04.612306 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b148ec-620b-429a-86aa-9e00a0092f01 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:04.612591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b148ec-620b-429a-86aa-9e00a0092f01 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:04.612867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b148ec-620b-429a-86aa-9e00a0092f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:04.631285 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-f6b148ec-620b-429a-86aa-9e00a0092f01 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:04.632481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-f6b148ec-620b-429a-86aa-9e00a0092f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:04.660020 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:04.667320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9f317-c310-4ab9-84df-a334c1e5478b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:04.699713 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 401/815] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:04.700782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b148ec-620b-429a-86aa-9e00a0092f01 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:04.711640 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ef4ca-8652-474e-af1f-09cfe275a510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:04.712292 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ef4ca-8652-474e-af1f-09cfe275a510 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:04.712574 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ef4ca-8652-474e-af1f-09cfe275a510 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:04.712847 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ef4ca-8652-474e-af1f-09cfe275a510 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:04.724599 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ef4ca-8652-474e-af1f-09cfe275a510 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:04.725615 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ef4ca-8652-474e-af1f-09cfe275a510 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:04.743335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-5b3ef4ca-8652-474e-af1f-09cfe275a510 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:04.743335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:04.743335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:04.743335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:04.743335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:04.743335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:04.743335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:04.743335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:04.743335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:04.743335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:04.743335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:04.743335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:04.743335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:27:04.743335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:27:04.743335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:27:04.744235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:27:04.744235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:04.744235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:04.744235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:04.744235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:04.744235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:04.744235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:04.744235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:04.744235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:04.744235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:04.744235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:04.744235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:04.744235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:04.744235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:04.744235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:04.744235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:04.744235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:04.749772 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:27:04.749772 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:27:04.749772 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:04.749772 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:04.749772 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 415/816] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:04.749772 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ef4ca-8652-474e-af1f-09cfe275a510 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:04.786356 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:04.819641 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b148ec-620b-429a-86aa-9e00a0092f01 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:04.854524 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ef4ca-8652-474e-af1f-09cfe275a510 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:04.856653 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5cc025-76be-45a0-a880-bdbd50d8aa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:04.857348 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cc025-76be-45a0-a880-bdbd50d8aa34 None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:04.857797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cc025-76be-45a0-a880-bdbd50d8aa34 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:04.858142 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cc025-76be-45a0-a880-bdbd50d8aa34 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:04.858860 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5cc025-76be-45a0-a880-bdbd50d8aa34 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:04.860782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5cc025-76be-45a0-a880-bdbd50d8aa34 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:04.889593 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 402/817] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:27:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:27:04.889593 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5cc025-76be-45a0-a880-bdbd50d8aa34 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:04.912305 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429e582-7a25-4152-89e1-aab60d194cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:04.913057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429e582-7a25-4152-89e1-aab60d194cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:04.913336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429e582-7a25-4152-89e1-aab60d194cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:04.913637 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429e582-7a25-4152-89e1-aab60d194cc4 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:04.914453 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429e582-7a25-4152-89e1-aab60d194cc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:04.914809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429e582-7a25-4152-89e1-aab60d194cc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:04.916550 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429e582-7a25-4152-89e1-aab60d194cc4 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:04.937954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 416/818] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:27:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:04.939608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429e582-7a25-4152-89e1-aab60d194cc4 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:04.944839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:05.039874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5cc025-76be-45a0-a880-bdbd50d8aa34 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:05.075497 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429e582-7a25-4152-89e1-aab60d194cc4 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:05.077939 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:05.087263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf8981-430d-4882-819f-816205c80ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:05.088376 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf8981-430d-4882-819f-816205c80ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:05.088994 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf8981-430d-4882-819f-816205c80ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:05.089531 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf8981-430d-4882-819f-816205c80ea4 None admin] PATH_INFO: `/v3/services/network` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:05.090343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf8981-430d-4882-819f-816205c80ea4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:05.091705 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf8981-430d-4882-819f-816205c80ea4 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:05.102419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-7baf8981-430d-4882-819f-816205c80ea4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 04:27:05.102419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:05.102419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:27:05.102419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:27:05.102419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:27:05.102419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:27:05.102419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:27:05.102419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:05.102419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 04:27:05.102419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:05.102419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:27:05.102419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:05.102419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:05.102419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:05.102419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:05.102419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:05.102419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:05.103420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:05.103420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:05.103420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:05.103420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:05.103420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:05.103420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:05.103420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:27:05.103420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:27:05.103420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:27:05.103420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:27:05.103420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:05.103420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:05.103420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:05.103420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:05.103420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:05.103420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:05.104235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:05.104235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:05.104235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:05.104235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:05.104235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:05.104235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:05.104235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:05.104235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:05.104235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:05.104235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:05.104235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:27:05.104235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:05.104235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 04:27:05.104235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:05.104235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 403/819] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:27:04 2022] GET /identity/v3/services/network => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:05.105027 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf8981-430d-4882-819f-816205c80ea4 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:05.126754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ad4ae-1c71-4395-baec-6d372fff7faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:05.127740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ad4ae-1c71-4395-baec-6d372fff7faf None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:05.128080 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ad4ae-1c71-4395-baec-6d372fff7faf None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:05.128359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ad4ae-1c71-4395-baec-6d372fff7faf None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:05.129056 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ad4ae-1c71-4395-baec-6d372fff7faf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:05.129364 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ad4ae-1c71-4395-baec-6d372fff7faf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:05.130869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ad4ae-1c71-4395-baec-6d372fff7faf None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:05.144205 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 417/820] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:05.144879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ad4ae-1c71-4395-baec-6d372fff7faf None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:05.228269 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:05.236527 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf8981-430d-4882-819f-816205c80ea4 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:05.277216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ad4ae-1c71-4395-baec-6d372fff7faf None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:05.279450 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b1896-5c52-4e68-9d41-cb0142ab268c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:05.280421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b1896-5c52-4e68-9d41-cb0142ab268c None admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:05.280902 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b1896-5c52-4e68-9d41-cb0142ab268c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:05.281404 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b1896-5c52-4e68-9d41-cb0142ab268c None admin] PATH_INFO: `/v3/projects/f37a5f3140ec431da8ea14568654114a/users/3a721708c7e443e799349add27de8f84/roles/cc86b64693b6417e90ad9361460574b0` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:05.301676 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b1896-5c52-4e68-9d41-cb0142ab268c None admin] RBAC: Authorizing `identity:create_grant(project_id=f37a5f3140ec431da8ea14568654114a, user_id=3a721708c7e443e799349add27de8f84, role_id=cc86b64693b6417e90ad9361460574b0)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:05.303798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b1896-5c52-4e68-9d41-cb0142ab268c None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:05.314904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a2ca6-8b1c-4300-9902-ed9c64fd2bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:05.315542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a2ca6-8b1c-4300-9902-ed9c64fd2bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:05.315823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a2ca6-8b1c-4300-9902-ed9c64fd2bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:05.316099 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a2ca6-8b1c-4300-9902-ed9c64fd2bf8 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:05.316814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a2ca6-8b1c-4300-9902-ed9c64fd2bf8 None admin] RBAC: Adding query filter params (name=network) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:05.317287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a2ca6-8b1c-4300-9902-ed9c64fd2bf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:05.318498 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a2ca6-8b1c-4300-9902-ed9c64fd2bf8 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:05.323442 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 404/821] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Aug 4 04:27:05 2022] PUT /identity/v3/projects/f37a5f3140ec431da8ea14568654114a/users/3a721708c7e443e799349add27de8f84/roles/cc86b64693b6417e90ad9361460574b0 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:27:05.324541 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b1896-5c52-4e68-9d41-cb0142ab268c None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:05.330506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 418/822] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Aug 4 04:27:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:05.338552 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a2ca6-8b1c-4300-9902-ed9c64fd2bf8 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:05.387617 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:05.463435 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b1896-5c52-4e68-9d41-cb0142ab268c None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:05.508955 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a9df2-0afe-43e5-a933-01e0162a0a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:05.516066 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a9df2-0afe-43e5-a933-01e0162a0a88 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:05.517187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a9df2-0afe-43e5-a933-01e0162a0a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:05.517713 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a9df2-0afe-43e5-a933-01e0162a0a88 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:05.527112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a9df2-0afe-43e5-a933-01e0162a0a88 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:05.528909 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a9df2-0afe-43e5-a933-01e0162a0a88 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:05.543206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-e86a9df2-0afe-43e5-a933-01e0162a0a88 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 04:27:05.543206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:05.543206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:05.543206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:05.543206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:05.543206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:05.543206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:05.543206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:05.543206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:05.543206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:05.543206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:05.543206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:05.543206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:27:05.543206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:27:05.543206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:27:05.543206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:27:05.544224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:05.544224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:05.544224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:27:05.544224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:05.544224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:27:05.544224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:05.544224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:05.544224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:05.544224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:05.544224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:05.544224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:05.544224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:05.544224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:05.544224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:05.544224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:05.544224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:05.544224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:05.544224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:05.545407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:05.545407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:05.545407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:27:05.545407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:27:05.545407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:27:05.545407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:27:05.545407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:27:05.545407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:27:05.545407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 04:27:05.545407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:05.548926 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 405/823] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Aug 4 04:27:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:05.554909 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a9df2-0afe-43e5-a933-01e0162a0a88 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:05.599603 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a2ca6-8b1c-4300-9902-ed9c64fd2bf8 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:05.633966 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:05.647962 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd55bde-041d-48a1-95ca-7024cc72d04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:05.649012 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd55bde-041d-48a1-95ca-7024cc72d04f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:05.649301 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd55bde-041d-48a1-95ca-7024cc72d04f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:05.649640 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd55bde-041d-48a1-95ca-7024cc72d04f None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:05.650338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd55bde-041d-48a1-95ca-7024cc72d04f None admin] RBAC: Adding query filter params (type=network) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:05.650645 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd55bde-041d-48a1-95ca-7024cc72d04f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:05.652022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd55bde-041d-48a1-95ca-7024cc72d04f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:05.663513 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 419/824] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Aug 4 04:27:05 2022] GET /identity/v3/services?type=network => generated 350 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:05.670403 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd55bde-041d-48a1-95ca-7024cc72d04f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:05.737017 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:05.742092 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a9df2-0afe-43e5-a933-01e0162a0a88 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:05.815076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd55bde-041d-48a1-95ca-7024cc72d04f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:05.819491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1209aa-b140-4c8e-be6a-6dc72dff44fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:05.820404 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1209aa-b140-4c8e-be6a-6dc72dff44fb None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:05.820856 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1209aa-b140-4c8e-be6a-6dc72dff44fb None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:05.821296 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1209aa-b140-4c8e-be6a-6dc72dff44fb None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:05.822284 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1209aa-b140-4c8e-be6a-6dc72dff44fb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:05.822782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1209aa-b140-4c8e-be6a-6dc72dff44fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:05.830102 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1209aa-b140-4c8e-be6a-6dc72dff44fb None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:05.851766 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d761ef1-1348-4b46-9874-5dcd8945dde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:05.852386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d761ef1-1348-4b46-9874-5dcd8945dde2 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:05.852664 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d761ef1-1348-4b46-9874-5dcd8945dde2 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:05.852935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d761ef1-1348-4b46-9874-5dcd8945dde2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:05.853674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d761ef1-1348-4b46-9874-5dcd8945dde2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:05.855047 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d761ef1-1348-4b46-9874-5dcd8945dde2 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:05.866401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 406/825] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Aug 4 04:27:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:05.872099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1209aa-b140-4c8e-be6a-6dc72dff44fb None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:05.896993 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 420/826] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:27:05 2022] POST /identity/v3/endpoints => generated 337 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:27:06.016672 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1209aa-b140-4c8e-be6a-6dc72dff44fb None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:06.060863 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7203f90-aa29-4119-9ccd-c1697bc5df40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:06.061493 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7203f90-aa29-4119-9ccd-c1697bc5df40 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:06.061798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7203f90-aa29-4119-9ccd-c1697bc5df40 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:06.062122 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7203f90-aa29-4119-9ccd-c1697bc5df40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:06.091966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:06.165239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7203f90-aa29-4119-9ccd-c1697bc5df40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:06.166712 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7203f90-aa29-4119-9ccd-c1697bc5df40 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:06.279274 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 407/827] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:27:05 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 04:27:06.289975 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d761ef1-1348-4b46-9874-5dcd8945dde2 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:06.351659 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:06.413225 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d761ef1-1348-4b46-9874-5dcd8945dde2 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:06.426467 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c134f-e457-494c-83c7-c2ece02fcd43 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:06.426753 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c134f-e457-494c-83c7-c2ece02fcd43 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:06.427026 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c134f-e457-494c-83c7-c2ece02fcd43 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:06.428490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 408/828] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:27:06.436049 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed49ac7-7ee1-46a5-88d4-6ee197f27b8e None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:06.436331 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed49ac7-7ee1-46a5-88d4-6ee197f27b8e None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:06.436600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed49ac7-7ee1-46a5-88d4-6ee197f27b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:06.460263 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561d7dc-8a0d-4053-a78d-f83dbb9fd62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:06.461962 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561d7dc-8a0d-4053-a78d-f83dbb9fd62d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:06.462479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561d7dc-8a0d-4053-a78d-f83dbb9fd62d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:06.462735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-9ed49ac7-7ee1-46a5-88d4-6ee197f27b8e None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:06.463161 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561d7dc-8a0d-4053-a78d-f83dbb9fd62d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:06.463914 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-9ed49ac7-7ee1-46a5-88d4-6ee197f27b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:06.479904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3561d7dc-8a0d-4053-a78d-f83dbb9fd62d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:06.481714 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3561d7dc-8a0d-4053-a78d-f83dbb9fd62d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:06.501188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-3561d7dc-8a0d-4053-a78d-f83dbb9fd62d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:06.501188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:06.501188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:06.501188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:06.501188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:06.501188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:06.501188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:06.501188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:06.501188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:06.501188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:06.501188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:06.501188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:06.501188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:27:06.501188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:27:06.501188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:27:06.502238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:27:06.502238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:06.502238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:06.502238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:06.502238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:06.502238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:06.502238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:06.502238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:06.502238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:06.502238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:06.502238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:06.502238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:06.502238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:06.502238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:06.502238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:06.502238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:06.502238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:06.503125 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:27:06.503125 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:27:06.503125 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:06.503125 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:06.505833 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 421/829] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:06.512873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561d7dc-8a0d-4053-a78d-f83dbb9fd62d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:06.537622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:06.557058 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 409/830] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:06 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 04:27:06.589772 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed49ac7-7ee1-46a5-88d4-6ee197f27b8e None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:06.617707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:06.625323 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561d7dc-8a0d-4053-a78d-f83dbb9fd62d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:06.663525 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f16945-0c95-4c2c-8f1c-13377c5f8822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:06.664150 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f16945-0c95-4c2c-8f1c-13377c5f8822 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:06.664430 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f16945-0c95-4c2c-8f1c-13377c5f8822 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:06.664703 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f16945-0c95-4c2c-8f1c-13377c5f8822 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:06.665459 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f16945-0c95-4c2c-8f1c-13377c5f8822 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:06.665792 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f16945-0c95-4c2c-8f1c-13377c5f8822 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:06.667216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f16945-0c95-4c2c-8f1c-13377c5f8822 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:06.682244 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 422/831] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:06.687572 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f16945-0c95-4c2c-8f1c-13377c5f8822 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:06.729570 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed49ac7-7ee1-46a5-88d4-6ee197f27b8e None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:06.767726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f123fc7-6df7-4100-bd44-5c19e5e9a699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:06.768483 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f123fc7-6df7-4100-bd44-5c19e5e9a699 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:06.768768 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f123fc7-6df7-4100-bd44-5c19e5e9a699 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:06.769046 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f123fc7-6df7-4100-bd44-5c19e5e9a699 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:06.786372 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f123fc7-6df7-4100-bd44-5c19e5e9a699 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:27:06.793625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-1f123fc7-6df7-4100-bd44-5c19e5e9a699 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:27:06.793625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:06.793625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:06.793625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:06.793625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:06.793625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:06.793625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:06.793625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:06.793625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:06.793625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:06.793625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:06.793625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:06.793625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:27:06.793625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:27:06.793625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:27:06.793625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:27:06.794680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:27:06.794680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:27:06.794680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:27:06.794680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:27:06.794680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:06.794680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:06.794680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:06.794680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:06.794680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:06.794680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:06.794680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:06.794680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:06.794680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:06.794680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:06.794680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:06.794680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:06.795755 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:06.795755 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:06.795755 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:06.795755 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:06.795755 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:27:06.795755 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:27:06.795755 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:27:06.795755 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:27:06.795755 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:27:06.795755 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:27:06.795755 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:27:06.795755 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:06.798912 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 410/832] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:27:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:06.806546 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f123fc7-6df7-4100-bd44-5c19e5e9a699 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:06.815445 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f16945-0c95-4c2c-8f1c-13377c5f8822 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:06.851901 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd617b8-f450-4f30-893c-1c9960fc1ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:06.852702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd617b8-f450-4f30-893c-1c9960fc1ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:06.853153 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd617b8-f450-4f30-893c-1c9960fc1ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:06.853532 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd617b8-f450-4f30-893c-1c9960fc1ab6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:06.857706 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:06.863688 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd617b8-f450-4f30-893c-1c9960fc1ab6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:06.865267 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd617b8-f450-4f30-893c-1c9960fc1ab6 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:06.880783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-7fd617b8-f450-4f30-893c-1c9960fc1ab6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:06.880783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:06.880783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:06.880783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:06.880783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:06.880783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:06.880783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:06.880783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:06.880783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:06.880783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:06.880783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:06.880783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:06.880783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:27:06.880783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:27:06.880783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:27:06.881872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:27:06.881872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:06.881872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:06.881872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:06.881872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:06.881872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:06.881872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:06.881872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:06.881872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:06.881872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:06.881872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:06.881872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:06.881872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:06.881872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:06.881872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:06.881872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:06.881872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:06.882958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:27:06.882958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:27:06.882958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:27:06.882958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:27:06.882958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:27:06.882958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:27:06.882958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:06.882958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:06.884673 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 423/833] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:27:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:06.890824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd617b8-f450-4f30-893c-1c9960fc1ab6 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:06.918441 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:06.934270 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f123fc7-6df7-4100-bd44-5c19e5e9a699 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:06.982182 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c1870-0ad2-49b1-8723-2627e909f76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:06.983072 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c1870-0ad2-49b1-8723-2627e909f76d None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:06.983507 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c1870-0ad2-49b1-8723-2627e909f76d None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:06.983931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c1870-0ad2-49b1-8723-2627e909f76d None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:06.985847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c1870-0ad2-49b1-8723-2627e909f76d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:06.986041 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c1870-0ad2-49b1-8723-2627e909f76d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:06.987368 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c1870-0ad2-49b1-8723-2627e909f76d None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:07.005041 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 411/834] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:27:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:07.005655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c1870-0ad2-49b1-8723-2627e909f76d None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:07.022316 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd617b8-f450-4f30-893c-1c9960fc1ab6 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:07.059647 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcee7db-5dea-4dbb-a31c-874e950d4c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:07.060619 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcee7db-5dea-4dbb-a31c-874e950d4c53 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:07.061190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcee7db-5dea-4dbb-a31c-874e950d4c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:07.061707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcee7db-5dea-4dbb-a31c-874e950d4c53 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:07.062646 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcee7db-5dea-4dbb-a31c-874e950d4c53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:07.063321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcee7db-5dea-4dbb-a31c-874e950d4c53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:07.065348 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcee7db-5dea-4dbb-a31c-874e950d4c53 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:07.085480 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 424/835] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:27:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:07.086814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcee7db-5dea-4dbb-a31c-874e950d4c53 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:07.111917 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c1870-0ad2-49b1-8723-2627e909f76d None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:07.119454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:07.148367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41255215-45d5-45af-9744-e091165d2ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:07.148964 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41255215-45d5-45af-9744-e091165d2ccc None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:07.149248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41255215-45d5-45af-9744-e091165d2ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:07.149517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41255215-45d5-45af-9744-e091165d2ccc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:07.160869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41255215-45d5-45af-9744-e091165d2ccc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:07.162324 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41255215-45d5-45af-9744-e091165d2ccc None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:07.179504 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:07.179835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-41255215-45d5-45af-9744-e091165d2ccc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:07.179835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:07.179835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:07.179835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:07.179835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:07.179835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:07.179835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:07.179835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:07.179835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:07.179835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:07.179835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:07.179835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:07.179835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:27:07.179835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:27:07.179835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:27:07.180690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:27:07.180690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:07.180690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:07.180690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:07.180690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:07.180690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:07.180690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:07.180690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:07.180690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:07.180690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:07.180690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:07.180690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:07.180690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:07.180690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:07.180690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:07.180690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:07.180690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:07.181602 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:27:07.181602 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:27:07.181602 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:07.181602 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:07.181602 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 412/836] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:07.187698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41255215-45d5-45af-9744-e091165d2ccc None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:07.222490 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcee7db-5dea-4dbb-a31c-874e950d4c53 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:07.260980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ecf11-ef65-4aae-9824-293992b7c241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:07.262095 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ecf11-ef65-4aae-9824-293992b7c241 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:07.262572 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ecf11-ef65-4aae-9824-293992b7c241 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:07.263137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ecf11-ef65-4aae-9824-293992b7c241 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:07.263994 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ecf11-ef65-4aae-9824-293992b7c241 None admin] RBAC: Adding query filter params (role.id=4a6517e5890a49718303007dac0756c7, scope.project.id=f37a5f3140ec431da8ea14568654114a, user.id=caa2901451aa4554bdfcac1bddbc620a) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:07.264311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ecf11-ef65-4aae-9824-293992b7c241 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:07.265935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ecf11-ef65-4aae-9824-293992b7c241 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:07.276818 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 425/837] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:27:07 2022] GET /identity/v3/role_assignments?user.id=caa2901451aa4554bdfcac1bddbc620a&scope.project.id=f37a5f3140ec431da8ea14568654114a&role.id=4a6517e5890a49718303007dac0756c7 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:07.276818 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066f205-ed0f-4d21-9bd2-c0bb28baad42 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:07.276818 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066f205-ed0f-4d21-9bd2-c0bb28baad42 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:07.276818 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066f205-ed0f-4d21-9bd2-c0bb28baad42 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:07.276818 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 426/838] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:07.283840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e1e5d-1d04-411a-84ef-8933da149c92 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:07.284269 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e1e5d-1d04-411a-84ef-8933da149c92 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:07.284616 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e1e5d-1d04-411a-84ef-8933da149c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:07.305358 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-4e6e1e5d-1d04-411a-84ef-8933da149c92 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:07.306963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4e6e1e5d-1d04-411a-84ef-8933da149c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:07.308769 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41255215-45d5-45af-9744-e091165d2ccc None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:07.352398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ff183-ebae-4226-96de-a9523424b364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:07.353156 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ff183-ebae-4226-96de-a9523424b364 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:07.353504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ff183-ebae-4226-96de-a9523424b364 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:07.354003 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ff183-ebae-4226-96de-a9523424b364 None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:07.354788 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ff183-ebae-4226-96de-a9523424b364 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:07.355160 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ff183-ebae-4226-96de-a9523424b364 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:07.356621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ff183-ebae-4226-96de-a9523424b364 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:07.376203 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 413/839] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:27:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:07.379514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ff183-ebae-4226-96de-a9523424b364 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:07.401798 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 427/840] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 04:27:07.414151 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:07.425271 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e1e5d-1d04-411a-84ef-8933da149c92 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:07.520181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ff183-ebae-4226-96de-a9523424b364 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:07.527447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:07.567069 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65114f40-6e06-4fc5-9366-97d0475f152a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:07.567690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65114f40-6e06-4fc5-9366-97d0475f152a None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:07.568219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65114f40-6e06-4fc5-9366-97d0475f152a None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:07.568219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65114f40-6e06-4fc5-9366-97d0475f152a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:07.571595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e1e5d-1d04-411a-84ef-8933da149c92 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:07.575962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65114f40-6e06-4fc5-9366-97d0475f152a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:07.577416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65114f40-6e06-4fc5-9366-97d0475f152a None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:07.590104 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-65114f40-6e06-4fc5-9366-97d0475f152a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 04:27:07.590104 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:07.590104 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:07.590104 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:07.590104 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:07.590104 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:07.590104 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:07.590104 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:07.590104 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:07.590104 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:07.590104 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:07.590104 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:07.590104 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:27:07.590104 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:27:07.590104 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:27:07.590104 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:27:07.591079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:07.591079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:07.591079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:27:07.591079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:07.591079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:27:07.591079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:07.591079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:07.591079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:07.591079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:07.591079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:07.591079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:07.591079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:07.591079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:07.591079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:07.591079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:07.591079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:07.591079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:07.591079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:07.592007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:07.592007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:07.592007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:27:07.592007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:27:07.592007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:27:07.592007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:27:07.592007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:27:07.592007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:27:07.592007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 04:27:07.592007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:07.592007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 414/841] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:27:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:07.597565 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65114f40-6e06-4fc5-9366-97d0475f152a None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:07.620634 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a3d0b-bedb-409a-8f77-fb29e5e72f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:07.622101 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a3d0b-bedb-409a-8f77-fb29e5e72f29 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:07.622382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a3d0b-bedb-409a-8f77-fb29e5e72f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:07.622652 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a3d0b-bedb-409a-8f77-fb29e5e72f29 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:07.623558 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a3d0b-bedb-409a-8f77-fb29e5e72f29 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:07.626109 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a3d0b-bedb-409a-8f77-fb29e5e72f29 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:07.638767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-4e5a3d0b-bedb-409a-8f77-fb29e5e72f29 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 04:27:07.638767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:07.638767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:27:07.638767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:27:07.638767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:27:07.638767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:27:07.638767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:27:07.638767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:07.638767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 04:27:07.638767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:07.638767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:27:07.638767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:07.638767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:07.638767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:07.638767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:07.638767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:07.638767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:07.640832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:07.640832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:07.640832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:07.640832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:07.640832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:07.640832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:07.640832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:27:07.640832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:27:07.640832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:27:07.640832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:27:07.640832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:07.640832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:07.640832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:07.640832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:07.640832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:07.640832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:07.643512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:07.643512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:07.643512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:07.643512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:07.643512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:07.643512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:07.643512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:07.643512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:07.643512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:07.643512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:07.643512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:27:07.643512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:07.643512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 04:27:07.643512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:07.643512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 428/842] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Aug 4 04:27:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:07.651569 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a3d0b-bedb-409a-8f77-fb29e5e72f29 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:07.690732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:07.715611 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65114f40-6e06-4fc5-9366-97d0475f152a None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:07.754255 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb390f-bd03-4e3d-ab66-7a028cf4933a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:07.755002 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb390f-bd03-4e3d-ab66-7a028cf4933a None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:07.755287 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb390f-bd03-4e3d-ab66-7a028cf4933a None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:07.755563 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb390f-bd03-4e3d-ab66-7a028cf4933a None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:07.756361 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb390f-bd03-4e3d-ab66-7a028cf4933a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:07.756671 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb390f-bd03-4e3d-ab66-7a028cf4933a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:07.757849 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a3d0b-bedb-409a-8f77-fb29e5e72f29 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:07.758126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb390f-bd03-4e3d-ab66-7a028cf4933a None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:07.781786 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 415/843] 10.222.0.47 () {56 vars in 1135 bytes} [Thu Aug 4 04:27:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:07.781786 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865e7a5-57be-4880-b179-1259c1977f05 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:07.781786 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865e7a5-57be-4880-b179-1259c1977f05 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:07.781786 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865e7a5-57be-4880-b179-1259c1977f05 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:07.781786 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 416/844] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:07.784906 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865e7a5-57be-4880-b179-1259c1977f05 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:07.798913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83c307-3ce5-420c-8106-6b7e9f74ad68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:07.799534 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83c307-3ce5-420c-8106-6b7e9f74ad68 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:07.799816 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83c307-3ce5-420c-8106-6b7e9f74ad68 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:07.800090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83c307-3ce5-420c-8106-6b7e9f74ad68 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:07.800769 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d83c307-3ce5-420c-8106-6b7e9f74ad68 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:07.801077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d83c307-3ce5-420c-8106-6b7e9f74ad68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:07.802469 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d83c307-3ce5-420c-8106-6b7e9f74ad68 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:07.806930 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 429/845] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Aug 4 04:27:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:07.808066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d490a-70d2-4a4a-ae8d-693b6f490156 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:07.808352 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d490a-70d2-4a4a-ae8d-693b6f490156 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:07.808624 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d490a-70d2-4a4a-ae8d-693b6f490156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:07.814998 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:07.827752 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-cf1d490a-70d2-4a4a-ae8d-693b6f490156 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:07.829503 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-cf1d490a-70d2-4a4a-ae8d-693b6f490156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:07.900697 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 430/846] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:07.901960 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d490a-70d2-4a4a-ae8d-693b6f490156 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:07.909590 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865e7a5-57be-4880-b179-1259c1977f05 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:07.948146 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1e6cc-759b-40fe-bf7a-77d6705fe625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:07.948864 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1e6cc-759b-40fe-bf7a-77d6705fe625 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:07.949148 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1e6cc-759b-40fe-bf7a-77d6705fe625 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:07.949422 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1e6cc-759b-40fe-bf7a-77d6705fe625 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:07.953261 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:08.022843 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d490a-70d2-4a4a-ae8d-693b6f490156 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:08.026453 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1e6cc-759b-40fe-bf7a-77d6705fe625 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:08.027895 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1e6cc-759b-40fe-bf7a-77d6705fe625 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:08.059567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:08.067328 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c01a2a-78e8-4b18-9bcd-164e2a41f3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:08.067953 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c01a2a-78e8-4b18-9bcd-164e2a41f3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:08.068237 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c01a2a-78e8-4b18-9bcd-164e2a41f3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:08.068514 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c01a2a-78e8-4b18-9bcd-164e2a41f3f4 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:08.069206 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c01a2a-78e8-4b18-9bcd-164e2a41f3f4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:08.069514 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c01a2a-78e8-4b18-9bcd-164e2a41f3f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:08.070922 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c01a2a-78e8-4b18-9bcd-164e2a41f3f4 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:08.075621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 431/847] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Aug 4 04:27:07 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:08.076404 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c01a2a-78e8-4b18-9bcd-164e2a41f3f4 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:08.168669 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 417/848] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:27:07 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 04:27:08.170836 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1e6cc-759b-40fe-bf7a-77d6705fe625 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:08.194520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c01a2a-78e8-4b18-9bcd-164e2a41f3f4 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:08.212021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:08.234854 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5299659-7c13-4da0-87dd-aaaf59b1ff87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:08.235600 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5299659-7c13-4da0-87dd-aaaf59b1ff87 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:08.235945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5299659-7c13-4da0-87dd-aaaf59b1ff87 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:08.236225 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5299659-7c13-4da0-87dd-aaaf59b1ff87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:08.248684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5299659-7c13-4da0-87dd-aaaf59b1ff87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:27:08.255115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-c5299659-7c13-4da0-87dd-aaaf59b1ff87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:27:08.255115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:08.255115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:08.255115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:08.255115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:08.255115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:08.255115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:08.255115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:08.255115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:08.255115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:08.255115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:08.255115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:08.255115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:27:08.255115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:27:08.255115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:27:08.255115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:27:08.256032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:27:08.256032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:27:08.256032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:27:08.256032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:27:08.256032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:08.256032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:08.256032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:08.256032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:08.256032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:08.256032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:08.256032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:08.256032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:08.256032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:08.256032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:08.256032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:08.256032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:08.256855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:08.256855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:08.256855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:08.256855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:08.256855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:27:08.256855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:27:08.256855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:27:08.256855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:27:08.256855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:27:08.256855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:27:08.256855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:27:08.256855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:08.264069 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 432/849] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Aug 4 04:27:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:08.265640 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5299659-7c13-4da0-87dd-aaaf59b1ff87 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:08.294122 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1e6cc-759b-40fe-bf7a-77d6705fe625 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:08.332993 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd070a30-6e85-4aab-b946-362cfd72b49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:08.333721 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd070a30-6e85-4aab-b946-362cfd72b49d None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:08.334305 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd070a30-6e85-4aab-b946-362cfd72b49d None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:08.334657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd070a30-6e85-4aab-b946-362cfd72b49d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:08.345900 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:08.347992 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd070a30-6e85-4aab-b946-362cfd72b49d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:08.349509 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd070a30-6e85-4aab-b946-362cfd72b49d None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:08.368292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-bd070a30-6e85-4aab-b946-362cfd72b49d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:08.368292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:08.368292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:08.368292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:08.368292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:08.368292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:08.368292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:08.368292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:08.368292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:08.368292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:08.368292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:08.368292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:08.368292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:27:08.368292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:27:08.368292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:27:08.369446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:27:08.369446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:08.369446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:08.369446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:08.369446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:08.369446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:08.369446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:08.369446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:08.369446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:08.369446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:08.369446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:08.369446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:08.369446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:08.369446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:08.369446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:08.369446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:08.369446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:08.370579 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:27:08.370579 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:27:08.370579 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:08.370579 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:08.377908 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 418/850] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:08.380904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd070a30-6e85-4aab-b946-362cfd72b49d None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:08.417115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5299659-7c13-4da0-87dd-aaaf59b1ff87 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:08.456757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af866a0c-59c3-431f-81da-850ae61a5674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:08.457478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af866a0c-59c3-431f-81da-850ae61a5674 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:08.457850 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af866a0c-59c3-431f-81da-850ae61a5674 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:08.458195 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af866a0c-59c3-431f-81da-850ae61a5674 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:08.458989 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af866a0c-59c3-431f-81da-850ae61a5674 None admin] RBAC: Adding query filter params (name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:08.459365 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af866a0c-59c3-431f-81da-850ae61a5674 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:08.460996 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af866a0c-59c3-431f-81da-850ae61a5674 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:08.474507 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 433/851] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Aug 4 04:27:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:08.477098 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af866a0c-59c3-431f-81da-850ae61a5674 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:08.497199 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:08.524162 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd070a30-6e85-4aab-b946-362cfd72b49d None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:08.563779 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbbf3cf-898f-4e2c-976c-c5849b9a630b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:08.564112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbf3cf-898f-4e2c-976c-c5849b9a630b None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:08.564368 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbf3cf-898f-4e2c-976c-c5849b9a630b None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:08.564643 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbf3cf-898f-4e2c-976c-c5849b9a630b None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:08.565608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbbf3cf-898f-4e2c-976c-c5849b9a630b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:08.565789 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbbf3cf-898f-4e2c-976c-c5849b9a630b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:08.567142 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbbf3cf-898f-4e2c-976c-c5849b9a630b None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:08.588347 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 419/852] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:08.589086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbbf3cf-898f-4e2c-976c-c5849b9a630b None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:08.631098 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af866a0c-59c3-431f-81da-850ae61a5674 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:08.655753 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:08.681105 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c5d59-1093-4969-a64a-b09364e96b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:08.682110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c5d59-1093-4969-a64a-b09364e96b00 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:08.682812 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c5d59-1093-4969-a64a-b09364e96b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:08.683779 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c5d59-1093-4969-a64a-b09364e96b00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:08.698703 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c5d59-1093-4969-a64a-b09364e96b00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:08.701288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c5d59-1093-4969-a64a-b09364e96b00 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:08.702868 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbbf3cf-898f-4e2c-976c-c5849b9a630b None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:08.722448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-007c5d59-1093-4969-a64a-b09364e96b00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:08.722448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:08.722448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:08.722448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:08.722448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:08.722448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:08.722448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:08.722448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:08.722448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:08.722448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:08.722448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:08.722448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:08.722448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:27:08.722448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:27:08.722448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:27:08.723376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:27:08.723376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:08.723376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:08.723376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:08.723376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:08.723376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:08.723376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:08.723376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:08.723376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:08.723376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:08.723376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:08.723376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:08.723376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:08.723376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:08.723376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:08.723376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:08.723376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:08.724258 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:27:08.724258 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:27:08.724258 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:08.724258 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:08.731547 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 434/853] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:08.733263 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c5d59-1093-4969-a64a-b09364e96b00 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:08.755050 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edb214-dd36-432d-a7ef-2fed40110233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:08.756265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edb214-dd36-432d-a7ef-2fed40110233 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:08.756722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edb214-dd36-432d-a7ef-2fed40110233 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:08.757170 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edb214-dd36-432d-a7ef-2fed40110233 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:08.767018 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76edb214-dd36-432d-a7ef-2fed40110233 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:08.769372 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76edb214-dd36-432d-a7ef-2fed40110233 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:08.784988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-76edb214-dd36-432d-a7ef-2fed40110233 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:08.784988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:08.784988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:08.784988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:08.784988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:08.784988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:08.784988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:08.784988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:08.784988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:08.784988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:08.784988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:08.784988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:08.784988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:27:08.784988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:27:08.784988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:27:08.785935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:27:08.785935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:08.785935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:08.785935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:08.785935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:08.785935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:08.785935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:08.785935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:08.785935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:08.785935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:08.785935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:08.785935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:08.785935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:08.785935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:08.785935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:08.785935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:08.785935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:08.786813 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:27:08.786813 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:27:08.786813 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:27:08.786813 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:27:08.786813 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:27:08.786813 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:27:08.786813 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:08.786813 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:08.790909 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 420/854] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:27:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:08.793038 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edb214-dd36-432d-a7ef-2fed40110233 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:08.808284 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:08.855504 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c5d59-1093-4969-a64a-b09364e96b00 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:08.895059 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45241074-7e47-449b-a5e2-5c27d061cc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:08.896113 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45241074-7e47-449b-a5e2-5c27d061cc8e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:08.896601 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45241074-7e47-449b-a5e2-5c27d061cc8e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:08.897112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45241074-7e47-449b-a5e2-5c27d061cc8e None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:08.898082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45241074-7e47-449b-a5e2-5c27d061cc8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:08.898644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45241074-7e47-449b-a5e2-5c27d061cc8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:08.900377 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45241074-7e47-449b-a5e2-5c27d061cc8e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:08.912681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edb214-dd36-432d-a7ef-2fed40110233 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:08.914737 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 435/855] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:08.921327 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45241074-7e47-449b-a5e2-5c27d061cc8e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:08.955598 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6446a-b535-4c57-a26e-c80ffbdee7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:08.956589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6446a-b535-4c57-a26e-c80ffbdee7e8 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:08.957112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:08.958375 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6446a-b535-4c57-a26e-c80ffbdee7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:08.958860 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6446a-b535-4c57-a26e-c80ffbdee7e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:08.959987 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a6446a-b535-4c57-a26e-c80ffbdee7e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:08.960495 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a6446a-b535-4c57-a26e-c80ffbdee7e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:08.963249 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a6446a-b535-4c57-a26e-c80ffbdee7e8 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:08.983481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 421/856] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:27:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:08.985085 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6446a-b535-4c57-a26e-c80ffbdee7e8 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:09.041629 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45241074-7e47-449b-a5e2-5c27d061cc8e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:09.061467 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:09.084978 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007d8bb-33c3-4694-8099-de32804bd6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:09.085745 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007d8bb-33c3-4694-8099-de32804bd6f4 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:09.086024 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007d8bb-33c3-4694-8099-de32804bd6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:09.086296 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007d8bb-33c3-4694-8099-de32804bd6f4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:09.096129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4007d8bb-33c3-4694-8099-de32804bd6f4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:09.098120 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4007d8bb-33c3-4694-8099-de32804bd6f4 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:09.103884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6446a-b535-4c57-a26e-c80ffbdee7e8 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:09.112905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-4007d8bb-33c3-4694-8099-de32804bd6f4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 04:27:09.112905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:09.112905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:09.112905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:09.112905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:09.112905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:09.112905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:09.112905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:09.112905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:09.112905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:09.112905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:09.112905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:09.112905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:27:09.112905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:27:09.112905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:27:09.112905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:27:09.113969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:09.113969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:09.113969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:27:09.113969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:09.113969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:27:09.113969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:09.113969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:09.113969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:09.113969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:09.113969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:09.113969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:09.113969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:09.113969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:09.113969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:09.113969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:09.113969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:09.113969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:09.113969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:09.114911 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:09.114911 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:09.114911 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:27:09.114911 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:27:09.114911 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:27:09.114911 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:27:09.114911 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:27:09.114911 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:27:09.114911 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 04:27:09.114911 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:09.117626 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 436/857] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Aug 4 04:27:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:09.121058 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007d8bb-33c3-4694-8099-de32804bd6f4 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:09.155600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e80c5b-2266-41c3-ae0e-c417bd933662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:09.156216 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e80c5b-2266-41c3-ae0e-c417bd933662 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:09.156494 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e80c5b-2266-41c3-ae0e-c417bd933662 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:09.156802 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e80c5b-2266-41c3-ae0e-c417bd933662 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:09.157707 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e80c5b-2266-41c3-ae0e-c417bd933662 None admin] RBAC: Adding query filter params (role.id=4a6517e5890a49718303007dac0756c7, scope.project.id=f37a5f3140ec431da8ea14568654114a, user.id=b1e539006166434790823b839bd745d5) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:09.158016 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e80c5b-2266-41c3-ae0e-c417bd933662 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:09.159656 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e80c5b-2266-41c3-ae0e-c417bd933662 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:09.173605 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 422/858] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:27:08 2022] GET /identity/v3/role_assignments?user.id=b1e539006166434790823b839bd745d5&scope.project.id=f37a5f3140ec431da8ea14568654114a&role.id=4a6517e5890a49718303007dac0756c7 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:09.229528 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:09.240258 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007d8bb-33c3-4694-8099-de32804bd6f4 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:09.278316 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b991abe-9f85-4522-9714-3f0e841cc418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:09.278945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b991abe-9f85-4522-9714-3f0e841cc418 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:09.279220 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b991abe-9f85-4522-9714-3f0e841cc418 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:09.279491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b991abe-9f85-4522-9714-3f0e841cc418 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:09.280297 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b991abe-9f85-4522-9714-3f0e841cc418 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:09.280671 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b991abe-9f85-4522-9714-3f0e841cc418 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:09.282332 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b991abe-9f85-4522-9714-3f0e841cc418 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:09.308784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 437/859] 10.222.0.47 () {56 vars in 1149 bytes} [Thu Aug 4 04:27:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:09.310226 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e80c5b-2266-41c3-ae0e-c417bd933662 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:09.423121 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e80c5b-2266-41c3-ae0e-c417bd933662 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:09.443374 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425df8ee-9eac-475d-b488-8830944b989f None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:09.443692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425df8ee-9eac-475d-b488-8830944b989f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:09.443976 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425df8ee-9eac-475d-b488-8830944b989f None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:09.445425 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 438/860] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:27:09.450359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587d4f1-08f8-4ab9-9ce7-8d64a84c4029 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:09.450638 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587d4f1-08f8-4ab9-9ce7-8d64a84c4029 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:09.450905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587d4f1-08f8-4ab9-9ce7-8d64a84c4029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:09.469071 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-d587d4f1-08f8-4ab9-9ce7-8d64a84c4029 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:09.470310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d587d4f1-08f8-4ab9-9ce7-8d64a84c4029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:09.471997 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2ecaf-bf5a-4133-bb64-bfa707586120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:09.473185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2ecaf-bf5a-4133-bb64-bfa707586120 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:09.473748 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2ecaf-bf5a-4133-bb64-bfa707586120 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:09.474228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2ecaf-bf5a-4133-bb64-bfa707586120 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:09.489901 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:09.552657 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 439/861] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 04:27:09.562830 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2ecaf-bf5a-4133-bb64-bfa707586120 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:09.565165 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2ecaf-bf5a-4133-bb64-bfa707586120 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:09.576226 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587d4f1-08f8-4ab9-9ce7-8d64a84c4029 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:09.676477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:09.705616 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587d4f1-08f8-4ab9-9ce7-8d64a84c4029 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:09.711402 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 423/862] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:27:09 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 04:27:09.718980 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2ecaf-bf5a-4133-bb64-bfa707586120 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:09.748264 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:09.749797 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a93428-80a5-49ef-b152-20fbb4546048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:09.751471 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a93428-80a5-49ef-b152-20fbb4546048 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:09.752122 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a93428-80a5-49ef-b152-20fbb4546048 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:09.753413 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a93428-80a5-49ef-b152-20fbb4546048 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:09.754298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a93428-80a5-49ef-b152-20fbb4546048 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:09.755842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a93428-80a5-49ef-b152-20fbb4546048 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:09.766134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-f9a93428-80a5-49ef-b152-20fbb4546048 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 04:27:09.766134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:09.766134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:27:09.766134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:27:09.766134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:27:09.766134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:27:09.766134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:27:09.766134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:09.766134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 04:27:09.766134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:09.766134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:27:09.766134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:09.766134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:09.766134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:09.766134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:09.766134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:09.767400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:09.767400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:09.767400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:09.767400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:09.767400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:09.767400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:09.767400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:09.767400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:27:09.767400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:27:09.767400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:27:09.767400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:27:09.767400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:09.767400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:09.767400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:09.767400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:09.767400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:09.768359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:09.768359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:09.768359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:09.768359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:09.768359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:09.768359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:09.768359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:09.768359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:09.768359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:09.768359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:09.768359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:09.768359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:27:09.768359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:09.768359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 04:27:09.768359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:09.770784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 440/863] 10.222.0.47 () {56 vars in 1132 bytes} [Thu Aug 4 04:27:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:09.782108 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a93428-80a5-49ef-b152-20fbb4546048 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:09.826577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2ecaf-bf5a-4133-bb64-bfa707586120 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:09.876630 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c8c49-cf1e-4201-965e-bc33004dd38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:09.877976 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c8c49-cf1e-4201-965e-bc33004dd38a None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:09.878415 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c8c49-cf1e-4201-965e-bc33004dd38a None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:09.879080 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c8c49-cf1e-4201-965e-bc33004dd38a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:09.895038 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c8c49-cf1e-4201-965e-bc33004dd38a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:09.895892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a93428-80a5-49ef-b152-20fbb4546048 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:09.896853 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c8c49-cf1e-4201-965e-bc33004dd38a None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:09.918337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-3d7c8c49-cf1e-4201-965e-bc33004dd38a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:09.918337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:09.918337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:09.918337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:09.918337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:09.918337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:09.918337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:09.918337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:09.918337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:09.918337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:09.918337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:09.918337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:09.918337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:27:09.918337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:27:09.918337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:27:09.919460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:27:09.919460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:09.919460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:09.919460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:09.919460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:09.919460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:09.919460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:09.919460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:09.919460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:09.919460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:09.919460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:09.919460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:09.919460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:09.919460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:09.919460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:09.919460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:09.919460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:09.920681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:27:09.920681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:27:09.920681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:09.920681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:09.923457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 424/864] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:09.929206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c8c49-cf1e-4201-965e-bc33004dd38a None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:09.940971 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1e74a-370c-4279-99a6-0e6734f01b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:09.942009 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1e74a-370c-4279-99a6-0e6734f01b02 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:09.942512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1e74a-370c-4279-99a6-0e6734f01b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:09.943075 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1e74a-370c-4279-99a6-0e6734f01b02 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:09.944039 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb1e74a-370c-4279-99a6-0e6734f01b02 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:09.944552 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb1e74a-370c-4279-99a6-0e6734f01b02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:09.946222 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb1e74a-370c-4279-99a6-0e6734f01b02 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:09.951808 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 441/865] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Aug 4 04:27:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:09.960328 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1e74a-370c-4279-99a6-0e6734f01b02 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:09.973448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:10.039057 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:10.059080 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c8c49-cf1e-4201-965e-bc33004dd38a None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:10.068728 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1e74a-370c-4279-99a6-0e6734f01b02 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:10.098632 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b20296-3342-4924-9df9-6b210a19c82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:10.099691 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b20296-3342-4924-9df9-6b210a19c82d None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:10.100230 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b20296-3342-4924-9df9-6b210a19c82d None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:10.100826 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b20296-3342-4924-9df9-6b210a19c82d None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:10.101757 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b20296-3342-4924-9df9-6b210a19c82d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:10.102289 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b20296-3342-4924-9df9-6b210a19c82d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:10.103957 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eaefb2-fc00-4106-9fef-bd33bd649103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:10.104274 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b20296-3342-4924-9df9-6b210a19c82d None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:10.104572 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eaefb2-fc00-4106-9fef-bd33bd649103 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:10.104848 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eaefb2-fc00-4106-9fef-bd33bd649103 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:10.105118 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eaefb2-fc00-4106-9fef-bd33bd649103 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:10.105814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eaefb2-fc00-4106-9fef-bd33bd649103 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:10.106281 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eaefb2-fc00-4106-9fef-bd33bd649103 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:10.107625 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eaefb2-fc00-4106-9fef-bd33bd649103 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:10.113579 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 442/866] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Aug 4 04:27:09 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:10.124366 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c355486-a862-48f4-a685-2d8c160de0e5 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:10.124940 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c355486-a862-48f4-a685-2d8c160de0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:10.125439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c355486-a862-48f4-a685-2d8c160de0e5 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:10.127037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 425/867] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:27:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:10.131676 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 443/868] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:27:10.135915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b20296-3342-4924-9df9-6b210a19c82d None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:10.137506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c227f-b169-4c8d-be1e-d3071af6f37b None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:10.137823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c227f-b169-4c8d-be1e-d3071af6f37b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:10.138093 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c227f-b169-4c8d-be1e-d3071af6f37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:10.161328 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-894c227f-b169-4c8d-be1e-d3071af6f37b None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:10.162873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-894c227f-b169-4c8d-be1e-d3071af6f37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:10.248180 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b20296-3342-4924-9df9-6b210a19c82d None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:10.256706 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:10.269581 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 444/869] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:10.287509 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c227f-b169-4c8d-be1e-d3071af6f37b None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:10.293276 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274ceed-9dea-4730-b5e6-7515533a2625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:10.294241 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274ceed-9dea-4730-b5e6-7515533a2625 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:10.294785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274ceed-9dea-4730-b5e6-7515533a2625 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:10.295509 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274ceed-9dea-4730-b5e6-7515533a2625 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:10.303524 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7274ceed-9dea-4730-b5e6-7515533a2625 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:10.305249 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7274ceed-9dea-4730-b5e6-7515533a2625 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:10.318190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-7274ceed-9dea-4730-b5e6-7515533a2625 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:10.318190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:10.318190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:10.318190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:10.318190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:10.318190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:10.318190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:10.318190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:10.318190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:10.318190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:10.318190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:10.318190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:10.318190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:27:10.318190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:27:10.318190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:27:10.319077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:27:10.319077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:10.319077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:10.319077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:10.319077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:10.319077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:10.319077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:10.319077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:10.319077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:10.319077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:10.319077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:10.319077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:10.319077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:10.319077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:10.319077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:10.319077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:10.319077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:10.319957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:27:10.319957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:27:10.319957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:27:10.319957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:27:10.319957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:27:10.319957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:27:10.319957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:10.319957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:10.321693 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 426/870] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:27:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:10.327347 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274ceed-9dea-4730-b5e6-7515533a2625 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:10.341925 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:10.416077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c227f-b169-4c8d-be1e-d3071af6f37b None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:10.439262 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274ceed-9dea-4730-b5e6-7515533a2625 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:10.461341 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd52d70-e129-457e-9e1a-76fe1bb0398b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:10.462358 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd52d70-e129-457e-9e1a-76fe1bb0398b None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:10.462841 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd52d70-e129-457e-9e1a-76fe1bb0398b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:10.463315 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd52d70-e129-457e-9e1a-76fe1bb0398b None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:10.464324 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd52d70-e129-457e-9e1a-76fe1bb0398b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:10.466133 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd52d70-e129-457e-9e1a-76fe1bb0398b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:10.475125 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703de116-8b89-42ab-b387-8bedceb08550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:10.475845 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703de116-8b89-42ab-b387-8bedceb08550 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:10.476072 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703de116-8b89-42ab-b387-8bedceb08550 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:10.476341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703de116-8b89-42ab-b387-8bedceb08550 None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:10.477117 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703de116-8b89-42ab-b387-8bedceb08550 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:10.477419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703de116-8b89-42ab-b387-8bedceb08550 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:10.478888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703de116-8b89-42ab-b387-8bedceb08550 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:10.486971 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 445/871] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Aug 4 04:27:10 2022] POST /identity/v3/services => generated 251 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 04:27:10.507320 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 427/872] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:27:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:10.517573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd52d70-e129-457e-9e1a-76fe1bb0398b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:10.593980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:10.653955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd52d70-e129-457e-9e1a-76fe1bb0398b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:10.690225 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1556b56-98d3-4cbd-9e3c-a7a608e8f4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:10.690862 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1556b56-98d3-4cbd-9e3c-a7a608e8f4f9 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:10.691114 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1556b56-98d3-4cbd-9e3c-a7a608e8f4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:10.691389 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1556b56-98d3-4cbd-9e3c-a7a608e8f4f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:10.692181 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1556b56-98d3-4cbd-9e3c-a7a608e8f4f9 None admin] RBAC: Adding query filter params (role.id=cc86b64693b6417e90ad9361460574b0, scope.project.id=f37a5f3140ec431da8ea14568654114a, user.id=3a721708c7e443e799349add27de8f84) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:10.692492 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1556b56-98d3-4cbd-9e3c-a7a608e8f4f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:10.693978 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1556b56-98d3-4cbd-9e3c-a7a608e8f4f9 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:10.700151 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 446/873] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:27:10 2022] GET /identity/v3/role_assignments?user.id=3a721708c7e443e799349add27de8f84&scope.project.id=f37a5f3140ec431da8ea14568654114a&role.id=cc86b64693b6417e90ad9361460574b0 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:11.354922 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be375cd1-d5de-4038-a9b5-0f8cd802365a None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:11.355284 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be375cd1-d5de-4038-a9b5-0f8cd802365a None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:11.355470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be375cd1-d5de-4038-a9b5-0f8cd802365a None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:11.357059 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 428/874] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:11.363475 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb13ab2-32a5-4470-b5eb-f9277a35e007 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:11.363831 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb13ab2-32a5-4470-b5eb-f9277a35e007 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:11.364148 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb13ab2-32a5-4470-b5eb-f9277a35e007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:11.392154 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-dfb13ab2-32a5-4470-b5eb-f9277a35e007 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:11.393444 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-dfb13ab2-32a5-4470-b5eb-f9277a35e007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:11.465334 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 447/875] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:11 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:11.487683 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be375cd1-d5de-4038-a9b5-0f8cd802365a None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:11.565575 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:11.607295 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be375cd1-d5de-4038-a9b5-0f8cd802365a None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:11.643783 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987150c-60cc-415a-9050-d599b0d16624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:11.644416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987150c-60cc-415a-9050-d599b0d16624 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:11.644693 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987150c-60cc-415a-9050-d599b0d16624 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:11.644961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987150c-60cc-415a-9050-d599b0d16624 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:11.645599 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987150c-60cc-415a-9050-d599b0d16624 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:11.646956 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987150c-60cc-415a-9050-d599b0d16624 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:11.656282 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-9987150c-60cc-415a-9050-d599b0d16624 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 04:27:11.656282 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:11.656282 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:27:11.656282 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:27:11.656282 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:27:11.656282 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:27:11.656282 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:27:11.656282 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:11.656282 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 04:27:11.656282 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:11.656282 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:27:11.656282 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:11.656282 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:11.656282 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:11.656282 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:11.656282 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:11.656282 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:11.657341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:11.657341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:11.657341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:11.657341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:11.657341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:11.657341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:11.657341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:27:11.657341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:27:11.657341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:27:11.657341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:27:11.657341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:11.657341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:11.657341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:11.657341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:11.657341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:11.657341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:11.658195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:11.658195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:11.658195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:11.658195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:11.658195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:11.658195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:11.658195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:11.658195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:11.658195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:11.658195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:11.658195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:27:11.658195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:11.658195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 04:27:11.658195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:11.658911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 429/876] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Aug 4 04:27:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:11.665420 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb13ab2-32a5-4470-b5eb-f9277a35e007 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:11.717528 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:11.776102 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb13ab2-32a5-4470-b5eb-f9277a35e007 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:11.811886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b81f0-b6c1-4db7-9d29-2124ed0daa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:11.812510 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b81f0-b6c1-4db7-9d29-2124ed0daa2d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:11.812829 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b81f0-b6c1-4db7-9d29-2124ed0daa2d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:11.813103 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b81f0-b6c1-4db7-9d29-2124ed0daa2d None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:11.813810 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7b81f0-b6c1-4db7-9d29-2124ed0daa2d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:11.814116 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7b81f0-b6c1-4db7-9d29-2124ed0daa2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:11.815477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7b81f0-b6c1-4db7-9d29-2124ed0daa2d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:11.820446 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 448/877] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Aug 4 04:27:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:11.826031 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987150c-60cc-415a-9050-d599b0d16624 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:11.942170 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987150c-60cc-415a-9050-d599b0d16624 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:11.972145 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:11.980374 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef67054-b35e-4ed3-9827-8bdc37cb3b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:11.980997 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef67054-b35e-4ed3-9827-8bdc37cb3b56 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:11.981277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef67054-b35e-4ed3-9827-8bdc37cb3b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:11.981566 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef67054-b35e-4ed3-9827-8bdc37cb3b56 None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:11.982252 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef67054-b35e-4ed3-9827-8bdc37cb3b56 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:11.982557 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef67054-b35e-4ed3-9827-8bdc37cb3b56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:11.983918 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef67054-b35e-4ed3-9827-8bdc37cb3b56 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:11.989810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 430/878] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Aug 4 04:27:11 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:12.332860 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61302c58-8ebb-4401-85db-1dc9ce6e9fda None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:12.333244 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61302c58-8ebb-4401-85db-1dc9ce6e9fda None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:12.333443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61302c58-8ebb-4401-85db-1dc9ce6e9fda None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:12.335043 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 449/879] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:12.340296 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c63a8-adc6-4bcd-ab45-f55975f8eb1b None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:12.340581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c63a8-adc6-4bcd-ab45-f55975f8eb1b None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:12.340858 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c63a8-adc6-4bcd-ab45-f55975f8eb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:12.359289 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-050c63a8-adc6-4bcd-ab45-f55975f8eb1b None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:12.361163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-050c63a8-adc6-4bcd-ab45-f55975f8eb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:12.434962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 431/880] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:12 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:12.457611 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61302c58-8ebb-4401-85db-1dc9ce6e9fda None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:12.580046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61302c58-8ebb-4401-85db-1dc9ce6e9fda None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:12.595237 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:12.615700 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7e126a-61ea-41ba-bbc6-60a6949255fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:12.616320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e126a-61ea-41ba-bbc6-60a6949255fc None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:12.616597 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e126a-61ea-41ba-bbc6-60a6949255fc None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:12.616865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e126a-61ea-41ba-bbc6-60a6949255fc None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:12.617571 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7e126a-61ea-41ba-bbc6-60a6949255fc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:12.618951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7e126a-61ea-41ba-bbc6-60a6949255fc None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:12.637735 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 450/881] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Aug 4 04:27:12 2022] POST /identity/v3/services => generated 269 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:27:12.670799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7cbe8-1229-44f2-8614-05841cc9390c None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:12.671360 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7cbe8-1229-44f2-8614-05841cc9390c None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:12.671360 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7cbe8-1229-44f2-8614-05841cc9390c None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:12.673426 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 432/882] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:12.679080 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6554158-ad61-41e4-b254-f346507d65e0 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:12.679363 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6554158-ad61-41e4-b254-f346507d65e0 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:12.679630 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6554158-ad61-41e4-b254-f346507d65e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:12.699352 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b6554158-ad61-41e4-b254-f346507d65e0 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:12.700579 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b6554158-ad61-41e4-b254-f346507d65e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:12.785121 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 451/883] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:12 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:12.807786 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7cbe8-1229-44f2-8614-05841cc9390c None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:12.899869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e69f31-254f-4c32-8a4b-5b6f7cc5b4e2 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:12.900114 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e69f31-254f-4c32-8a4b-5b6f7cc5b4e2 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:12.900350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e69f31-254f-4c32-8a4b-5b6f7cc5b4e2 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:12.902930 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 452/884] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:27:12.906989 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380be3fc-0778-4f55-b938-13baab63fa73 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:12.907370 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380be3fc-0778-4f55-b938-13baab63fa73 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:12.907786 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380be3fc-0778-4f55-b938-13baab63fa73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:12.923732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7cbe8-1229-44f2-8614-05841cc9390c None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:12.927228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-380be3fc-0778-4f55-b938-13baab63fa73 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:12.928429 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-380be3fc-0778-4f55-b938-13baab63fa73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:12.950474 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:12.969971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da76ff12-03ab-4818-aa69-4410de15d1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:12.970606 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76ff12-03ab-4818-aa69-4410de15d1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:12.970879 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76ff12-03ab-4818-aa69-4410de15d1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:12.971233 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76ff12-03ab-4818-aa69-4410de15d1d0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:12.971852 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da76ff12-03ab-4818-aa69-4410de15d1d0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:12.973199 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da76ff12-03ab-4818-aa69-4410de15d1d0 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:12.983201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-da76ff12-03ab-4818-aa69-4410de15d1d0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 04:27:12.983201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:12.983201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:27:12.983201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:27:12.983201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:27:12.983201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:27:12.983201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:27:12.983201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:12.983201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 04:27:12.983201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:12.983201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:27:12.983201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:12.983201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:12.983201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:12.983201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:12.983201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:12.983201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:12.984191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:12.984191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:12.984191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:12.984191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:12.984191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:12.984191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:12.984191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:27:12.984191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:27:12.984191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:27:12.984191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:27:12.984191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:12.984191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:12.984191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:12.984191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:12.984191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:12.984191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:12.986465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:12.986465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:12.986465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:12.986465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:12.986465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:12.986465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:12.986465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:12.986465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:12.986465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:12.986465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:12.986465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:27:12.986465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:12.986465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 04:27:12.986465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:12.989649 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 433/885] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Aug 4 04:27:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:12.991312 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da76ff12-03ab-4818-aa69-4410de15d1d0 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:13.025562 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 453/886] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:12 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 04:27:13.045417 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380be3fc-0778-4f55-b938-13baab63fa73 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:13.078223 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:13.108431 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da76ff12-03ab-4818-aa69-4410de15d1d0 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:13.145331 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed3b1e-1d2e-403e-8d5e-668c0c030f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:13.146317 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed3b1e-1d2e-403e-8d5e-668c0c030f09 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:13.146838 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed3b1e-1d2e-403e-8d5e-668c0c030f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:13.147315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed3b1e-1d2e-403e-8d5e-668c0c030f09 None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:13.148165 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed3b1e-1d2e-403e-8d5e-668c0c030f09 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:13.148639 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed3b1e-1d2e-403e-8d5e-668c0c030f09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:13.150273 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed3b1e-1d2e-403e-8d5e-668c0c030f09 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:13.155951 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 434/887] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Aug 4 04:27:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:13.161346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380be3fc-0778-4f55-b938-13baab63fa73 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:13.162726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed3b1e-1d2e-403e-8d5e-668c0c030f09 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:13.196378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dadb255-1b23-4b8b-a578-7cb59a0538f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:13.197030 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dadb255-1b23-4b8b-a578-7cb59a0538f0 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:13.197314 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dadb255-1b23-4b8b-a578-7cb59a0538f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:13.197611 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dadb255-1b23-4b8b-a578-7cb59a0538f0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:13.209290 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dadb255-1b23-4b8b-a578-7cb59a0538f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:27:13.214288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-8dadb255-1b23-4b8b-a578-7cb59a0538f0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 04:27:13.214288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:13.214288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:13.214288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:13.214288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:13.214288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:13.214288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:13.214288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:13.214288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:13.214288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:13.214288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:13.214288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:13.214288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:27:13.214288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:27:13.214288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:27:13.214288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:27:13.215304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:27:13.215304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:27:13.215304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:27:13.215304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:27:13.215304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:13.215304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:13.215304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:13.215304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:13.215304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:13.215304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:13.215304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:13.215304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:13.215304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:13.215304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:13.215304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:13.215304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:13.216129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:13.216129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:13.216129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:13.216129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:13.216129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:27:13.216129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:27:13.216129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:27:13.216129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:27:13.216129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:27:13.216129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:27:13.216129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 04:27:13.216129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:13.220648 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 454/888] 10.222.0.47 () {56 vars in 1120 bytes} [Thu Aug 4 04:27:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:13.222098 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dadb255-1b23-4b8b-a578-7cb59a0538f0 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:13.230648 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:13.286785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed3b1e-1d2e-403e-8d5e-668c0c030f09 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:13.321096 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:13.322434 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e4f0a-04a7-4213-92b0-716eb3357e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:13.324495 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e4f0a-04a7-4213-92b0-716eb3357e3e None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:13.324979 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e4f0a-04a7-4213-92b0-716eb3357e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:13.326208 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e4f0a-04a7-4213-92b0-716eb3357e3e None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:13.326995 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e4f0a-04a7-4213-92b0-716eb3357e3e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:13.327377 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e4f0a-04a7-4213-92b0-716eb3357e3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:13.330013 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e4f0a-04a7-4213-92b0-716eb3357e3e None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:13.335710 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dadb255-1b23-4b8b-a578-7cb59a0538f0 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:13.341346 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 435/889] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Aug 4 04:27:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:13.345316 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e4f0a-04a7-4213-92b0-716eb3357e3e None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:13.382206 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31f603-a335-4cf2-9b3f-ed6e0a62292d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:13.383667 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31f603-a335-4cf2-9b3f-ed6e0a62292d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:13.384154 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31f603-a335-4cf2-9b3f-ed6e0a62292d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:13.384600 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31f603-a335-4cf2-9b3f-ed6e0a62292d None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:13.385993 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31f603-a335-4cf2-9b3f-ed6e0a62292d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:13.386306 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31f603-a335-4cf2-9b3f-ed6e0a62292d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:13.387666 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31f603-a335-4cf2-9b3f-ed6e0a62292d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:13.412318 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 455/890] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Aug 4 04:27:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:13.417467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31f603-a335-4cf2-9b3f-ed6e0a62292d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:13.491885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e4f0a-04a7-4213-92b0-716eb3357e3e None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:13.510468 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:13.533754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31f603-a335-4cf2-9b3f-ed6e0a62292d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:13.541096 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895141e1-68b2-4964-813a-902ce30ada1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:13.542438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895141e1-68b2-4964-813a-902ce30ada1c None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:13.542871 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895141e1-68b2-4964-813a-902ce30ada1c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:13.543289 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895141e1-68b2-4964-813a-902ce30ada1c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:13.544690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895141e1-68b2-4964-813a-902ce30ada1c None admin] RBAC: Adding query filter params (interface=public, service_id=b3879945f198447d921aa9ee208d2e0e, region_id=RegionOne) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:13.545458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895141e1-68b2-4964-813a-902ce30ada1c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:13.547936 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895141e1-68b2-4964-813a-902ce30ada1c None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:13.554414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 436/891] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Aug 4 04:27:13 2022] GET /identity/v3/endpoints?service_id=b3879945f198447d921aa9ee208d2e0e&interface=public®ion_id=RegionOne => generated 194 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:13.559321 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895141e1-68b2-4964-813a-902ce30ada1c None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:13.575771 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6056c92-533b-4e68-a1cd-c0db1246028f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:13.576856 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6056c92-533b-4e68-a1cd-c0db1246028f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:13.577746 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6056c92-533b-4e68-a1cd-c0db1246028f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:13.578174 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6056c92-533b-4e68-a1cd-c0db1246028f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:13.593711 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6056c92-533b-4e68-a1cd-c0db1246028f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:13.596574 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6056c92-533b-4e68-a1cd-c0db1246028f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:13.615985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-f6056c92-533b-4e68-a1cd-c0db1246028f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:13.615985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:13.615985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:13.615985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:13.615985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:13.615985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:13.615985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:13.615985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:13.615985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:13.615985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:13.615985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:13.615985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:13.615985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:27:13.615985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:27:13.615985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:27:13.617076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:27:13.617076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:13.617076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:13.617076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:13.617076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:13.617076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:13.617076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:13.617076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:13.617076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:13.617076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:13.617076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:13.617076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:13.617076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:13.617076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:13.617076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:13.617076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:13.617076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:13.618203 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:27:13.618203 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:27:13.618203 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:13.618203 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:13.624728 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 456/892] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:13.627353 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6056c92-533b-4e68-a1cd-c0db1246028f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:13.632852 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:13.680713 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895141e1-68b2-4964-813a-902ce30ada1c None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:13.719330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b61faa-a579-4a13-9d48-2f37adf772dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:13.720317 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b61faa-a579-4a13-9d48-2f37adf772dc None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:13.720824 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b61faa-a579-4a13-9d48-2f37adf772dc None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:13.721295 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b61faa-a579-4a13-9d48-2f37adf772dc None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:13.722179 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b61faa-a579-4a13-9d48-2f37adf772dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:13.723899 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b61faa-a579-4a13-9d48-2f37adf772dc None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:13.730363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 437/893] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Aug 4 04:27:13 2022] GET /identity/v3/services => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:27:13.760758 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6056c92-533b-4e68-a1cd-c0db1246028f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:13.801230 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fa9b1-7058-4a18-87e0-189e70eb7783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:13.801990 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fa9b1-7058-4a18-87e0-189e70eb7783 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:13.802288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fa9b1-7058-4a18-87e0-189e70eb7783 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:13.802547 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fa9b1-7058-4a18-87e0-189e70eb7783 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:13.803237 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971fa9b1-7058-4a18-87e0-189e70eb7783 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:13.803547 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971fa9b1-7058-4a18-87e0-189e70eb7783 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:13.805016 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971fa9b1-7058-4a18-87e0-189e70eb7783 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:13.823871 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 457/894] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:13.825819 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b61faa-a579-4a13-9d48-2f37adf772dc None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:13.879246 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:13.940918 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b61faa-a579-4a13-9d48-2f37adf772dc None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:13.976509 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901af66-b353-48f7-a35b-c2b74dfe0c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:13.977158 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901af66-b353-48f7-a35b-c2b74dfe0c16 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:13.977443 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901af66-b353-48f7-a35b-c2b74dfe0c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:13.977744 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901af66-b353-48f7-a35b-c2b74dfe0c16 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:13.986399 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2901af66-b353-48f7-a35b-c2b74dfe0c16 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:13.987830 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2901af66-b353-48f7-a35b-c2b74dfe0c16 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:13.999358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-2901af66-b353-48f7-a35b-c2b74dfe0c16 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 04:27:13.999358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:13.999358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:13.999358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:13.999358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:13.999358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:13.999358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:13.999358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:13.999358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:13.999358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:13.999358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:13.999358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:13.999358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:27:13.999358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:27:13.999358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:27:13.999358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:27:14.000401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:14.000401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:14.000401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:27:14.000401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:14.000401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:27:14.000401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:14.000401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:14.000401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:14.000401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:14.000401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:14.000401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:14.000401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:14.000401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:14.000401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:14.000401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:14.000401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:14.000401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:14.000401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:14.001315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:14.001315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:14.001315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:27:14.001315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:27:14.001315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:27:14.001315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:27:14.001315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:27:14.001315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:27:14.001315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 04:27:14.001315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:14.005441 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 438/895] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Aug 4 04:27:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:14.006809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fa9b1-7058-4a18-87e0-189e70eb7783 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:14.072600 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:14.091199 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5969175-af68-4184-bdee-7cc822e47738 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:14.091502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5969175-af68-4184-bdee-7cc822e47738 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:14.091750 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5969175-af68-4184-bdee-7cc822e47738 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:14.098126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 439/896] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:27:14.099301 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ce6d9-b8ed-4693-81f9-0d365d1d9e30 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:14.099585 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ce6d9-b8ed-4693-81f9-0d365d1d9e30 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:14.099856 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ce6d9-b8ed-4693-81f9-0d365d1d9e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:14.114680 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fa9b1-7058-4a18-87e0-189e70eb7783 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:14.122600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-099ce6d9-b8ed-4693-81f9-0d365d1d9e30 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:14.123807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-099ce6d9-b8ed-4693-81f9-0d365d1d9e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:14.155907 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f567a9-4de3-4e1a-ad08-ca171dbbdc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:14.157070 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f567a9-4de3-4e1a-ad08-ca171dbbdc96 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:14.157354 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f567a9-4de3-4e1a-ad08-ca171dbbdc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:14.157666 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f567a9-4de3-4e1a-ad08-ca171dbbdc96 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:14.158461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f567a9-4de3-4e1a-ad08-ca171dbbdc96 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:14.158764 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f567a9-4de3-4e1a-ad08-ca171dbbdc96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:14.160187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f567a9-4de3-4e1a-ad08-ca171dbbdc96 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:14.178719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 458/897] 10.222.0.47 () {56 vars in 1149 bytes} [Thu Aug 4 04:27:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:14.184641 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f567a9-4de3-4e1a-ad08-ca171dbbdc96 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:14.214199 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 440/898] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:14 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:14.237005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ce6d9-b8ed-4693-81f9-0d365d1d9e30 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:14.282237 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:14.300855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f567a9-4de3-4e1a-ad08-ca171dbbdc96 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:14.325371 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:14.338075 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051cd1f1-a795-4661-9251-287f36dde645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:14.338693 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051cd1f1-a795-4661-9251-287f36dde645 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:14.339025 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051cd1f1-a795-4661-9251-287f36dde645 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:14.339235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051cd1f1-a795-4661-9251-287f36dde645 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:14.348515 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ce6d9-b8ed-4693-81f9-0d365d1d9e30 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:14.387146 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46cfef1-6c74-4ab6-bf27-c7197715835e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:14.388194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cfef1-6c74-4ab6-bf27-c7197715835e None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:14.388681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cfef1-6c74-4ab6-bf27-c7197715835e None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:14.389158 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cfef1-6c74-4ab6-bf27-c7197715835e None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:14.390188 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46cfef1-6c74-4ab6-bf27-c7197715835e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:14.392673 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46cfef1-6c74-4ab6-bf27-c7197715835e None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:14.417720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 441/899] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Aug 4 04:27:14 2022] POST /identity/v3/services => generated 241 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 04:27:14.418747 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051cd1f1-a795-4661-9251-287f36dde645 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:14.420618 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051cd1f1-a795-4661-9251-287f36dde645 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:14.549534 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 459/900] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:27:14 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 04:27:14.551088 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46cfef1-6c74-4ab6-bf27-c7197715835e None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:14.654083 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:14.671647 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46cfef1-6c74-4ab6-bf27-c7197715835e None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:14.708544 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1db77c-7ee3-4772-bb8c-88194b3e239f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:14.709196 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1db77c-7ee3-4772-bb8c-88194b3e239f None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:14.709458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1db77c-7ee3-4772-bb8c-88194b3e239f None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:14.709764 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1db77c-7ee3-4772-bb8c-88194b3e239f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:14.721426 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1db77c-7ee3-4772-bb8c-88194b3e239f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:14.722906 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1db77c-7ee3-4772-bb8c-88194b3e239f None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:14.738930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-2d1db77c-7ee3-4772-bb8c-88194b3e239f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:14.738930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:14.738930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:14.738930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:14.738930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:14.738930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:14.738930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:14.738930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:14.738930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:14.738930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:14.738930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:14.738930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:14.738930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:27:14.738930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:27:14.738930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:27:14.739866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:27:14.739866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:14.739866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:14.739866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:14.739866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:14.739866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:14.739866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:14.739866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:14.739866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:14.739866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:14.739866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:14.739866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:14.739866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:14.739866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:14.739866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:14.739866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:14.739866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:14.740729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:27:14.740729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:27:14.740729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:14.740729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:14.744809 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 442/901] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:14.746299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051cd1f1-a795-4661-9251-287f36dde645 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:14.772134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:14.803563 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46062bd4-cac3-45af-be26-dc754d9516d2 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:14.803876 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46062bd4-cac3-45af-be26-dc754d9516d2 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:14.804118 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46062bd4-cac3-45af-be26-dc754d9516d2 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:14.805742 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 443/902] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:27:14.810319 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97c12c-31df-4239-aa37-8f055ca9af42 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:14.810725 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97c12c-31df-4239-aa37-8f055ca9af42 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:14.811175 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97c12c-31df-4239-aa37-8f055ca9af42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:14.830614 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-da97c12c-31df-4239-aa37-8f055ca9af42 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:14.831902 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-da97c12c-31df-4239-aa37-8f055ca9af42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:14.856843 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051cd1f1-a795-4661-9251-287f36dde645 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:14.903288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9da94-5b6a-410a-b19c-0ecae0fdafa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:14.904079 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9da94-5b6a-410a-b19c-0ecae0fdafa7 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:14.904458 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9da94-5b6a-410a-b19c-0ecae0fdafa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:14.904832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9da94-5b6a-410a-b19c-0ecae0fdafa7 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:14.905662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9da94-5b6a-410a-b19c-0ecae0fdafa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:14.906071 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9da94-5b6a-410a-b19c-0ecae0fdafa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:14.907602 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9da94-5b6a-410a-b19c-0ecae0fdafa7 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:14.913932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 444/903] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:14.928233 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 460/904] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:27:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:14.929539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97c12c-31df-4239-aa37-8f055ca9af42 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:14.940615 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9da94-5b6a-410a-b19c-0ecae0fdafa7 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:15.009112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:15.035415 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:15.045300 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97c12c-31df-4239-aa37-8f055ca9af42 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:15.056816 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9da94-5b6a-410a-b19c-0ecae0fdafa7 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:15.084174 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8646f9-84f4-44e7-bc16-3b840bb3b621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:15.084820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8646f9-84f4-44e7-bc16-3b840bb3b621 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:15.085155 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8646f9-84f4-44e7-bc16-3b840bb3b621 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:15.085477 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8646f9-84f4-44e7-bc16-3b840bb3b621 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:15.093156 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8646f9-84f4-44e7-bc16-3b840bb3b621 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:15.095216 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8646f9-84f4-44e7-bc16-3b840bb3b621 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:15.099090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ed0e9-bbff-492f-b7e0-ba135d42075e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:15.100010 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ed0e9-bbff-492f-b7e0-ba135d42075e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:15.100485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ed0e9-bbff-492f-b7e0-ba135d42075e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:15.100966 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ed0e9-bbff-492f-b7e0-ba135d42075e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:15.101810 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ed0e9-bbff-492f-b7e0-ba135d42075e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:15.103416 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ed0e9-bbff-492f-b7e0-ba135d42075e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:15.107612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-7a8646f9-84f4-44e7-bc16-3b840bb3b621 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:15.107612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:15.107612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:15.107612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:15.107612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:15.107612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:15.107612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:15.107612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:15.107612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:15.107612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:15.107612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:15.107612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:15.107612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:27:15.107612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:27:15.107612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:27:15.108457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:27:15.108457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:15.108457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:15.108457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:15.108457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:15.108457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:15.108457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:15.108457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:15.108457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:15.108457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:15.108457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:15.108457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:15.108457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:15.108457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:15.108457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:15.108457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:15.108457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:15.109329 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:27:15.109329 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:27:15.109329 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:27:15.109329 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:27:15.109329 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:27:15.109329 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:27:15.109329 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:15.109329 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:15.109329 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 445/905] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:27:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:15.118000 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8646f9-84f4-44e7-bc16-3b840bb3b621 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:15.118145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-4f7ed0e9-bbff-492f-b7e0-ba135d42075e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 04:27:15.118145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:15.118145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:27:15.118145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:27:15.118145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:27:15.118145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:27:15.118145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:27:15.118145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:15.118145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 04:27:15.118145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:15.118145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:27:15.118145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:15.118145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:15.118145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:15.118145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:15.118145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:15.119112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:15.119112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:15.119112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:15.119112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:15.119112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:15.119112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:15.119112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:15.119112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:27:15.119112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:27:15.119112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:27:15.119112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:27:15.119112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:15.119112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:15.119112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:15.119112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:15.119112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:15.119926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:15.119926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:15.119926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:15.119926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:15.119926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:15.119926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:15.119926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:15.119926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:15.119926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:15.119926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:15.119926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:15.119926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:27:15.119926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:15.119926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 04:27:15.119926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:15.119926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 461/906] 10.222.0.47 () {56 vars in 1132 bytes} [Thu Aug 4 04:27:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:15.123213 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ed0e9-bbff-492f-b7e0-ba135d42075e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:15.239500 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8646f9-84f4-44e7-bc16-3b840bb3b621 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:15.273330 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ed0e9-bbff-492f-b7e0-ba135d42075e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:15.276352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:15.283802 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5a419-e138-4866-b372-f187abcd6d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:15.284705 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5a419-e138-4866-b372-f187abcd6d05 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:15.285174 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5a419-e138-4866-b372-f187abcd6d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:15.285718 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5a419-e138-4866-b372-f187abcd6d05 None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:15.286671 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b5a419-e138-4866-b372-f187abcd6d05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:15.287166 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b5a419-e138-4866-b372-f187abcd6d05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:15.288778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b5a419-e138-4866-b372-f187abcd6d05 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:15.307085 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 446/907] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:27:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:15.308694 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5a419-e138-4866-b372-f187abcd6d05 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:15.316805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751dc968-b3dc-45f3-822f-c4450e40ba4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:15.317433 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751dc968-b3dc-45f3-822f-c4450e40ba4f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:15.317741 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751dc968-b3dc-45f3-822f-c4450e40ba4f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:15.318012 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751dc968-b3dc-45f3-822f-c4450e40ba4f None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:15.318695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751dc968-b3dc-45f3-822f-c4450e40ba4f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:15.318996 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751dc968-b3dc-45f3-822f-c4450e40ba4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:15.320913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751dc968-b3dc-45f3-822f-c4450e40ba4f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:15.329644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 462/908] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Aug 4 04:27:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:15.334427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751dc968-b3dc-45f3-822f-c4450e40ba4f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:15.353515 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:15.429848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5a419-e138-4866-b372-f187abcd6d05 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:15.468029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13839a5d-7c33-4c2a-979f-07bbbd3aebac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:15.468853 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13839a5d-7c33-4c2a-979f-07bbbd3aebac None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:15.469998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13839a5d-7c33-4c2a-979f-07bbbd3aebac None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:15.470287 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13839a5d-7c33-4c2a-979f-07bbbd3aebac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:15.471220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13839a5d-7c33-4c2a-979f-07bbbd3aebac None admin] RBAC: Adding query filter params (role.id=3747dec438cd4349bd9e402694b88574, scope.project.id=f37a5f3140ec431da8ea14568654114a, user.id=3a721708c7e443e799349add27de8f84) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:15.471625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13839a5d-7c33-4c2a-979f-07bbbd3aebac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:15.473164 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13839a5d-7c33-4c2a-979f-07bbbd3aebac None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:15.480477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751dc968-b3dc-45f3-822f-c4450e40ba4f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:15.485804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 447/909] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:27:15 2022] GET /identity/v3/role_assignments?user.id=3a721708c7e443e799349add27de8f84&scope.project.id=f37a5f3140ec431da8ea14568654114a&role.id=3747dec438cd4349bd9e402694b88574 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:15.533674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e7c3f-4e63-47bb-a1b9-dbc5d8218637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:15.534599 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e7c3f-4e63-47bb-a1b9-dbc5d8218637 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:15.535032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e7c3f-4e63-47bb-a1b9-dbc5d8218637 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:15.535451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e7c3f-4e63-47bb-a1b9-dbc5d8218637 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:15.536388 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e7c3f-4e63-47bb-a1b9-dbc5d8218637 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:15.536859 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e7c3f-4e63-47bb-a1b9-dbc5d8218637 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:15.538476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e7c3f-4e63-47bb-a1b9-dbc5d8218637 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:15.549696 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 463/910] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Aug 4 04:27:15 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:15.552385 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13839a5d-7c33-4c2a-979f-07bbbd3aebac None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:15.616251 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:15.667653 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13839a5d-7c33-4c2a-979f-07bbbd3aebac None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:15.705194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332e373-843b-4b3f-99a3-257e149ad50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:15.706142 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332e373-843b-4b3f-99a3-257e149ad50d None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:15.706588 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332e373-843b-4b3f-99a3-257e149ad50d None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:15.707014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332e373-843b-4b3f-99a3-257e149ad50d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:15.708112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332e373-843b-4b3f-99a3-257e149ad50d None admin] RBAC: Adding query filter params (interface=public, service_id=2bbbdb0729d14f7a8b718f816dcc554f, region_id=RegionOne) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:15.708613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332e373-843b-4b3f-99a3-257e149ad50d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:15.710624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332e373-843b-4b3f-99a3-257e149ad50d None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:15.717017 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 448/911] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Aug 4 04:27:15 2022] GET /identity/v3/endpoints?service_id=2bbbdb0729d14f7a8b718f816dcc554f&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:15.723234 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e7c3f-4e63-47bb-a1b9-dbc5d8218637 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:15.804694 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:15.834831 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e7c3f-4e63-47bb-a1b9-dbc5d8218637 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:15.870913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37676a-d953-43a7-bdc3-e6d88891429d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:15.871557 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37676a-d953-43a7-bdc3-e6d88891429d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:15.871812 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37676a-d953-43a7-bdc3-e6d88891429d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:15.872074 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37676a-d953-43a7-bdc3-e6d88891429d None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:15.872735 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad37676a-d953-43a7-bdc3-e6d88891429d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:15.874137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad37676a-d953-43a7-bdc3-e6d88891429d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:15.876376 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487caead-ad4e-40f5-a6e3-a332293aed21 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:15.876884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487caead-ad4e-40f5-a6e3-a332293aed21 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:15.877368 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487caead-ad4e-40f5-a6e3-a332293aed21 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:15.878908 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 449/912] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:15.887463 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 464/913] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Aug 4 04:27:15 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:27:15.888921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9e619-ce09-40fc-9cf2-8413056dc556 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:15.889213 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9e619-ce09-40fc-9cf2-8413056dc556 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:15.889494 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9e619-ce09-40fc-9cf2-8413056dc556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:15.909475 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-d7d9e619-ce09-40fc-9cf2-8413056dc556 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:15.910757 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d7d9e619-ce09-40fc-9cf2-8413056dc556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:15.994428 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 450/914] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:16.017229 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37676a-d953-43a7-bdc3-e6d88891429d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:16.124404 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37676a-d953-43a7-bdc3-e6d88891429d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:16.160624 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff0b92e-48da-4401-a76e-5bf68fc10719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:16.161278 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0b92e-48da-4401-a76e-5bf68fc10719 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:16.161570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0b92e-48da-4401-a76e-5bf68fc10719 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:16.161871 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0b92e-48da-4401-a76e-5bf68fc10719 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:16.162570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff0b92e-48da-4401-a76e-5bf68fc10719 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:16.163999 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff0b92e-48da-4401-a76e-5bf68fc10719 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:16.174391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-dff0b92e-48da-4401-a76e-5bf68fc10719 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 04:27:16.174391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:16.174391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:27:16.174391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:27:16.174391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:27:16.174391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:27:16.174391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:27:16.174391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:16.174391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 04:27:16.174391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:16.174391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:27:16.174391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:16.174391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:16.174391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:16.174391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:16.174391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:16.174391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:16.175494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:16.175494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:16.175494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:16.175494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:16.175494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:16.175494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:16.175494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:27:16.175494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:27:16.175494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:27:16.175494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:27:16.175494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:16.175494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:16.175494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:16.175494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:16.175494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:16.175494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:16.176395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:16.176395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:16.176395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:16.176395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:16.176395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:16.176395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:16.176395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:16.176395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:16.176395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:16.176395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:16.176395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:27:16.176395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:16.176395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 04:27:16.176395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:16.176395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 465/915] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Aug 4 04:27:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:16.181858 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9e619-ce09-40fc-9cf2-8413056dc556 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:16.241141 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:16.299236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9e619-ce09-40fc-9cf2-8413056dc556 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:16.334430 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed169ac4-f0b3-452b-9bc1-d56d3ffb6daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:16.335050 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed169ac4-f0b3-452b-9bc1-d56d3ffb6daa None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:16.335333 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed169ac4-f0b3-452b-9bc1-d56d3ffb6daa None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:16.335609 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed169ac4-f0b3-452b-9bc1-d56d3ffb6daa None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:16.336293 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed169ac4-f0b3-452b-9bc1-d56d3ffb6daa None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:16.336638 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed169ac4-f0b3-452b-9bc1-d56d3ffb6daa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:16.338128 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed169ac4-f0b3-452b-9bc1-d56d3ffb6daa None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:16.344381 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 451/916] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Aug 4 04:27:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:16.350004 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff0b92e-48da-4401-a76e-5bf68fc10719 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:16.361884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:16.457403 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff0b92e-48da-4401-a76e-5bf68fc10719 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:16.493134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f2ea7-91ad-489e-ac51-931599fd6b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:16.493780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f2ea7-91ad-489e-ac51-931599fd6b81 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:16.494061 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f2ea7-91ad-489e-ac51-931599fd6b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:16.494330 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f2ea7-91ad-489e-ac51-931599fd6b81 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:16.495006 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f2ea7-91ad-489e-ac51-931599fd6b81 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:16.495311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f2ea7-91ad-489e-ac51-931599fd6b81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:16.496673 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f2ea7-91ad-489e-ac51-931599fd6b81 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:16.506420 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 466/917] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Aug 4 04:27:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:16.511684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed169ac4-f0b3-452b-9bc1-d56d3ffb6daa None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:16.627714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed169ac4-f0b3-452b-9bc1-d56d3ffb6daa None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:16.642403 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:16.666714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f006f5c-499c-4638-8335-36dd4d13e283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:16.667345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f006f5c-499c-4638-8335-36dd4d13e283 None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:16.667627 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f006f5c-499c-4638-8335-36dd4d13e283 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:16.667899 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f006f5c-499c-4638-8335-36dd4d13e283 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:16.668689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f006f5c-499c-4638-8335-36dd4d13e283 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:16.669187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55610ba2-71d8-4cb8-9dd9-78e100a4161a None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:16.669983 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55610ba2-71d8-4cb8-9dd9-78e100a4161a None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:16.670407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f006f5c-499c-4638-8335-36dd4d13e283 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:16.670677 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55610ba2-71d8-4cb8-9dd9-78e100a4161a None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:16.672580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 467/918] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:27:16.678193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff4c7c-5d06-47c5-96db-57e912d90628 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:16.678814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff4c7c-5d06-47c5-96db-57e912d90628 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:16.679304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff4c7c-5d06-47c5-96db-57e912d90628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:16.696494 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 452/919] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:27:16 2022] POST /identity/v3/endpoints => generated 346 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:27:16.701461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-1aff4c7c-5d06-47c5-96db-57e912d90628 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:16.703069 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1aff4c7c-5d06-47c5-96db-57e912d90628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:16.775114 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:16.791373 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 468/920] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 04:27:16.814619 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f006f5c-499c-4638-8335-36dd4d13e283 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:16.924907 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f006f5c-499c-4638-8335-36dd4d13e283 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:16.961247 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a4485-c6cf-4e75-afc3-51c3e5975fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:16.961919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a4485-c6cf-4e75-afc3-51c3e5975fca None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:16.962195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a4485-c6cf-4e75-afc3-51c3e5975fca None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:16.962466 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a4485-c6cf-4e75-afc3-51c3e5975fca None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:16.963098 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a4485-c6cf-4e75-afc3-51c3e5975fca None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:16.964502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a4485-c6cf-4e75-afc3-51c3e5975fca None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:16.974379 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-3b6a4485-c6cf-4e75-afc3-51c3e5975fca None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 04:27:16.974379 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:16.974379 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:27:16.974379 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:27:16.974379 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:27:16.974379 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:27:16.974379 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:27:16.974379 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:16.974379 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 04:27:16.974379 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:16.974379 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:27:16.974379 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:16.974379 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:16.974379 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:16.974379 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:16.974379 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:16.974379 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:16.975429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:16.975429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:16.975429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:16.975429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:16.975429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:16.975429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:16.975429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:27:16.975429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:27:16.975429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:27:16.975429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:27:16.975429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:16.975429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:16.975429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:16.975429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:16.975429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:16.975429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:16.976248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:16.976248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:16.976248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:16.976248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:16.976248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:16.976248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:16.976248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:16.976248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:16.976248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:16.976248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:16.976248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:27:16.976248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:16.976248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 04:27:16.976248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:16.976248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 453/921] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Aug 4 04:27:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:16.982641 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff4c7c-5d06-47c5-96db-57e912d90628 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:17.090220 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff4c7c-5d06-47c5-96db-57e912d90628 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:17.132593 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd8399-4285-434e-8e3d-21b63b5d4a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:17.133233 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd8399-4285-434e-8e3d-21b63b5d4a43 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:17.133511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd8399-4285-434e-8e3d-21b63b5d4a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:17.133814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd8399-4285-434e-8e3d-21b63b5d4a43 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:17.134516 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bd8399-4285-434e-8e3d-21b63b5d4a43 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:17.134817 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bd8399-4285-434e-8e3d-21b63b5d4a43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:17.136233 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bd8399-4285-434e-8e3d-21b63b5d4a43 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:17.142150 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 469/922] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Aug 4 04:27:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:17.147705 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a4485-c6cf-4e75-afc3-51c3e5975fca None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:17.183311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:17.268269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a4485-c6cf-4e75-afc3-51c3e5975fca None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:17.306059 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c531f0f-2ab9-48b9-af5c-689ad2628c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:17.306728 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c531f0f-2ab9-48b9-af5c-689ad2628c1a None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:17.307010 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c531f0f-2ab9-48b9-af5c-689ad2628c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:17.307279 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c531f0f-2ab9-48b9-af5c-689ad2628c1a None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:17.307953 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c531f0f-2ab9-48b9-af5c-689ad2628c1a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:17.308255 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c531f0f-2ab9-48b9-af5c-689ad2628c1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:17.309625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c531f0f-2ab9-48b9-af5c-689ad2628c1a None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:17.316321 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 454/923] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Aug 4 04:27:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:17.322281 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd8399-4285-434e-8e3d-21b63b5d4a43 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:17.384441 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:17.429954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd8399-4285-434e-8e3d-21b63b5d4a43 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:17.467431 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c6480-f02b-4c33-969d-efa57fc6ae68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:17.469165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c6480-f02b-4c33-969d-efa57fc6ae68 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:17.470000 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c6480-f02b-4c33-969d-efa57fc6ae68 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:17.470775 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c6480-f02b-4c33-969d-efa57fc6ae68 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:17.472499 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c6480-f02b-4c33-969d-efa57fc6ae68 None admin] RBAC: Adding query filter params (interface=public, service_id=908a24a25ee3442d8e6980c95454aeaa, region_id=RegionOne) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:17.473359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c6480-f02b-4c33-969d-efa57fc6ae68 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:17.476392 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c6480-f02b-4c33-969d-efa57fc6ae68 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:17.486638 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 470/924] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Aug 4 04:27:17 2022] GET /identity/v3/endpoints?service_id=908a24a25ee3442d8e6980c95454aeaa&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:17.497641 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c531f0f-2ab9-48b9-af5c-689ad2628c1a None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:17.602095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:17.607818 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c531f0f-2ab9-48b9-af5c-689ad2628c1a None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:17.645185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a720821-05be-4c9d-ab92-7114cfd14196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:17.645764 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a720821-05be-4c9d-ab92-7114cfd14196 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:17.646108 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a720821-05be-4c9d-ab92-7114cfd14196 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:17.646390 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a720821-05be-4c9d-ab92-7114cfd14196 None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:17.647050 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a720821-05be-4c9d-ab92-7114cfd14196 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:17.648424 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a720821-05be-4c9d-ab92-7114cfd14196 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:17.655008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 455/925] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Aug 4 04:27:17 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:27:17.710230 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d4445-26ff-458f-82df-59c2755a0f4e None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:17.710886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d4445-26ff-458f-82df-59c2755a0f4e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:17.711373 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d4445-26ff-458f-82df-59c2755a0f4e None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:17.713306 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 471/926] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:17.719467 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce9939-a502-47f1-97de-5c2c87d2e207 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:17.720106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce9939-a502-47f1-97de-5c2c87d2e207 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:17.720590 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce9939-a502-47f1-97de-5c2c87d2e207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:17.741340 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-b6ce9939-a502-47f1-97de-5c2c87d2e207 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:17.743121 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-b6ce9939-a502-47f1-97de-5c2c87d2e207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:17.825396 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 456/927] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:17.848994 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d4445-26ff-458f-82df-59c2755a0f4e None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:17.957060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d4445-26ff-458f-82df-59c2755a0f4e None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:17.980575 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:17.997296 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103c771-ec2e-42f2-8a11-807f8a1f920a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:17.998813 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103c771-ec2e-42f2-8a11-807f8a1f920a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:17.999308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103c771-ec2e-42f2-8a11-807f8a1f920a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:18.000379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103c771-ec2e-42f2-8a11-807f8a1f920a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:18.011578 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103c771-ec2e-42f2-8a11-807f8a1f920a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:27:18.016882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-c103c771-ec2e-42f2-8a11-807f8a1f920a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 04:27:18.016882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:18.016882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:18.016882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:18.016882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:18.016882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:18.016882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:18.016882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:18.016882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:18.016882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:18.016882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:18.016882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:18.016882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:27:18.016882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:27:18.016882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:27:18.016882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:27:18.017824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:27:18.017824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:27:18.017824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:27:18.017824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:27:18.017824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:18.017824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:18.017824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:18.017824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:18.017824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:18.017824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:18.017824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:18.017824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:18.017824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:18.017824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:18.017824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:18.017824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:18.018648 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:18.018648 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:18.018648 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:18.018648 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:18.018648 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:27:18.018648 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:27:18.018648 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:27:18.018648 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:27:18.018648 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:27:18.018648 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:27:18.018648 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 04:27:18.018648 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:18.021393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 472/928] 10.222.0.47 () {56 vars in 1120 bytes} [Thu Aug 4 04:27:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:18.027497 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce9939-a502-47f1-97de-5c2c87d2e207 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:18.121077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:18.132130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce9939-a502-47f1-97de-5c2c87d2e207 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:18.136508 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fada0f-8537-449a-921e-b9896d495f3c None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:18.136789 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fada0f-8537-449a-921e-b9896d495f3c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:18.137061 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fada0f-8537-449a-921e-b9896d495f3c None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:18.138657 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 473/929] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:18.146342 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abeff16-004f-4349-b9c0-0451039cd0b8 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:18.146737 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abeff16-004f-4349-b9c0-0451039cd0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:18.147147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abeff16-004f-4349-b9c0-0451039cd0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:18.166799 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7abeff16-004f-4349-b9c0-0451039cd0b8 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:18.168120 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7abeff16-004f-4349-b9c0-0451039cd0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:18.170386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d7ff8-4218-4d18-bd8b-d11b3d120b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:18.171253 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d7ff8-4218-4d18-bd8b-d11b3d120b58 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:18.172391 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d7ff8-4218-4d18-bd8b-d11b3d120b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:18.173351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d7ff8-4218-4d18-bd8b-d11b3d120b58 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:18.174364 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d7ff8-4218-4d18-bd8b-d11b3d120b58 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:18.174814 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d7ff8-4218-4d18-bd8b-d11b3d120b58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:18.176338 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d7ff8-4218-4d18-bd8b-d11b3d120b58 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:18.185694 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 457/930] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Aug 4 04:27:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:18.192069 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d7ff8-4218-4d18-bd8b-d11b3d120b58 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:18.257645 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 474/931] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:18.273392 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abeff16-004f-4349-b9c0-0451039cd0b8 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:18.323929 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d7ff8-4218-4d18-bd8b-d11b3d120b58 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:18.363388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca0eb7-5aba-419e-a2b2-25a9d955b33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:18.364466 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca0eb7-5aba-419e-a2b2-25a9d955b33a None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:18.365008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca0eb7-5aba-419e-a2b2-25a9d955b33a None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:18.365484 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca0eb7-5aba-419e-a2b2-25a9d955b33a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:18.381172 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca0eb7-5aba-419e-a2b2-25a9d955b33a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:18.383608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca0eb7-5aba-419e-a2b2-25a9d955b33a None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:18.397211 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abeff16-004f-4349-b9c0-0451039cd0b8 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:18.403555 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-72ca0eb7-5aba-419e-a2b2-25a9d955b33a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:18.403555 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:18.403555 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:18.403555 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:18.403555 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:18.403555 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:18.403555 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:18.403555 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:18.403555 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:18.403555 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:18.403555 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:18.403555 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:18.403555 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:27:18.403555 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:27:18.403555 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:27:18.404465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:27:18.404465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:18.404465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:18.404465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:18.404465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:18.404465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:18.404465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:18.404465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:18.404465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:18.404465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:18.404465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:18.404465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:18.404465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:18.404465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:18.404465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:18.404465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:18.404465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:18.405334 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:27:18.405334 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:27:18.405334 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:18.405334 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:18.412074 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 458/932] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:18.413754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:18.414092 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca0eb7-5aba-419e-a2b2-25a9d955b33a None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:18.418946 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:18.444388 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3f73c-6001-4ec1-8684-7bcb3a8461ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:18.445036 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3f73c-6001-4ec1-8684-7bcb3a8461ee None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:18.445312 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3f73c-6001-4ec1-8684-7bcb3a8461ee None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:18.445603 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3f73c-6001-4ec1-8684-7bcb3a8461ee None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:18.446234 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc3f73c-6001-4ec1-8684-7bcb3a8461ee None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:18.448023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc3f73c-6001-4ec1-8684-7bcb3a8461ee None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:18.460598 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-fbc3f73c-6001-4ec1-8684-7bcb3a8461ee None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 04:27:18.460598 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:18.460598 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:27:18.460598 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:27:18.460598 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:27:18.460598 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:27:18.460598 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:27:18.460598 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:18.460598 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 04:27:18.460598 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:18.460598 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:27:18.460598 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:18.460598 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:18.460598 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:18.460598 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:18.460598 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:18.461605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:18.461605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:18.461605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:18.461605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:18.461605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:18.461605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:18.461605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:18.461605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:27:18.461605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:27:18.461605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:27:18.461605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:27:18.461605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:18.461605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:18.461605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:18.461605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:18.461605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:18.462440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:18.462440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:18.462440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:18.462440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:18.462440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:18.462440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:18.462440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:18.462440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:18.462440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:18.462440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:18.462440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:18.462440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:27:18.462440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:18.462440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 04:27:18.462440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:18.465640 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 475/933] 10.222.0.47 () {56 vars in 1132 bytes} [Thu Aug 4 04:27:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:18.467642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3f73c-6001-4ec1-8684-7bcb3a8461ee None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:18.558289 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca0eb7-5aba-419e-a2b2-25a9d955b33a None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:18.589988 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3f73c-6001-4ec1-8684-7bcb3a8461ee None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:18.598856 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61826db-e495-4a36-9789-08f2191d68c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:18.600323 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61826db-e495-4a36-9789-08f2191d68c1 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:18.600877 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61826db-e495-4a36-9789-08f2191d68c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:18.601365 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61826db-e495-4a36-9789-08f2191d68c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:18.602448 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61826db-e495-4a36-9789-08f2191d68c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:18.602946 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61826db-e495-4a36-9789-08f2191d68c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:18.604628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61826db-e495-4a36-9789-08f2191d68c1 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:18.618749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 459/934] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:18.625530 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61826db-e495-4a36-9789-08f2191d68c1 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:18.633001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7507d-8c58-442a-839b-c0eef700c950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:18.633742 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7507d-8c58-442a-839b-c0eef700c950 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:18.634099 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7507d-8c58-442a-839b-c0eef700c950 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:18.634449 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7507d-8c58-442a-839b-c0eef700c950 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:18.635226 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7507d-8c58-442a-839b-c0eef700c950 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:18.635579 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7507d-8c58-442a-839b-c0eef700c950 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:18.637050 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7507d-8c58-442a-839b-c0eef700c950 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:18.648759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 476/935] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Aug 4 04:27:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:18.650790 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7507d-8c58-442a-839b-c0eef700c950 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:18.695504 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:18.697347 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:18.738663 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61826db-e495-4a36-9789-08f2191d68c1 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:18.765976 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7507d-8c58-442a-839b-c0eef700c950 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:18.775383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3808df43-5606-4753-965a-ea389edce9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:18.776475 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808df43-5606-4753-965a-ea389edce9de None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:18.776475 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808df43-5606-4753-965a-ea389edce9de None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:18.776685 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808df43-5606-4753-965a-ea389edce9de None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:18.785957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3808df43-5606-4753-965a-ea389edce9de None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:18.787445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3808df43-5606-4753-965a-ea389edce9de None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:18.801584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-3808df43-5606-4753-965a-ea389edce9de None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 04:27:18.801584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:18.801584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:18.801584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:18.801584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:18.801584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:18.801584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:18.801584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:18.801584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:18.801584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:18.801584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:18.801584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:18.801584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:27:18.801584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:27:18.801584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:27:18.801584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:27:18.802608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:18.802608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:18.802608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:27:18.802608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:18.802608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:27:18.802608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:18.802608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:18.802608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:18.802608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:18.802608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:18.802608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:18.802608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:18.802608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:18.802608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:18.802608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:18.802608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:18.802608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:18.802608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:18.803538 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:18.803538 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:18.803538 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:27:18.803538 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:27:18.803538 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:27:18.803538 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:27:18.803538 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:27:18.803538 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:27:18.803538 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 04:27:18.803538 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:18.807917 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 460/936] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Aug 4 04:27:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:18.809288 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3808df43-5606-4753-965a-ea389edce9de None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:18.813399 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3545e1-3aa3-45c8-87da-ba285c65bce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:18.814352 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3545e1-3aa3-45c8-87da-ba285c65bce0 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:18.814923 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3545e1-3aa3-45c8-87da-ba285c65bce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:18.815462 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3545e1-3aa3-45c8-87da-ba285c65bce0 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:18.816376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3545e1-3aa3-45c8-87da-ba285c65bce0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:18.816904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3545e1-3aa3-45c8-87da-ba285c65bce0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:18.819326 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3545e1-3aa3-45c8-87da-ba285c65bce0 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:18.832813 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 477/937] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Aug 4 04:27:18 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:18.834257 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3545e1-3aa3-45c8-87da-ba285c65bce0 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:18.915221 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3808df43-5606-4753-965a-ea389edce9de None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:18.939618 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:18.947801 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3545e1-3aa3-45c8-87da-ba285c65bce0 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:18.951804 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:18.951923 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82f39e-9dc2-4dda-ae47-73df8c5a4657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:18.952529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82f39e-9dc2-4dda-ae47-73df8c5a4657 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:18.952814 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82f39e-9dc2-4dda-ae47-73df8c5a4657 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:18.953791 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82f39e-9dc2-4dda-ae47-73df8c5a4657 None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:18.954886 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f82f39e-9dc2-4dda-ae47-73df8c5a4657 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:18.955201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f82f39e-9dc2-4dda-ae47-73df8c5a4657 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:18.956639 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f82f39e-9dc2-4dda-ae47-73df8c5a4657 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:18.981491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 461/938] 10.222.0.47 () {56 vars in 1149 bytes} [Thu Aug 4 04:27:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:18.982983 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d42e4-a9a8-4710-ac31-97267c23e12e None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:18.983277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d42e4-a9a8-4710-ac31-97267c23e12e None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:18.983542 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d42e4-a9a8-4710-ac31-97267c23e12e None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:18.989588 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 462/939] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:18.989588 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d42e4-a9a8-4710-ac31-97267c23e12e None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:18.990520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391648be-65d9-4753-8f7a-2365068a6bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:18.991248 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391648be-65d9-4753-8f7a-2365068a6bd1 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:18.991526 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391648be-65d9-4753-8f7a-2365068a6bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:18.991796 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391648be-65d9-4753-8f7a-2365068a6bd1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:18.992515 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391648be-65d9-4753-8f7a-2365068a6bd1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:18.993920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391648be-65d9-4753-8f7a-2365068a6bd1 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:19.013894 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 478/940] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:27:18 2022] POST /identity/v3/endpoints => generated 347 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:27:19.017684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc9d33-443a-43b0-a919-1b2649211468 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:19.017684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc9d33-443a-43b0-a919-1b2649211468 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:19.017684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc9d33-443a-43b0-a919-1b2649211468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:19.042320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-1efc9d33-443a-43b0-a919-1b2649211468 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:19.043836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1efc9d33-443a-43b0-a919-1b2649211468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:19.113519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d42e4-a9a8-4710-ac31-97267c23e12e None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:19.158321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 479/941] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:19.168012 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aff1d9-9391-4c48-965d-0507ed91be8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:19.168681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aff1d9-9391-4c48-965d-0507ed91be8a None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:19.168956 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aff1d9-9391-4c48-965d-0507ed91be8a None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:19.169227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aff1d9-9391-4c48-965d-0507ed91be8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:19.183203 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc9d33-443a-43b0-a919-1b2649211468 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:19.184424 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aff1d9-9391-4c48-965d-0507ed91be8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:19.187360 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aff1d9-9391-4c48-965d-0507ed91be8a None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:19.206189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-98aff1d9-9391-4c48-965d-0507ed91be8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:19.206189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:19.206189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:19.206189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:19.206189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:19.206189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:19.206189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:19.206189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:19.206189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:19.206189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:19.206189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:19.206189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:19.206189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:27:19.206189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:27:19.206189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:27:19.207225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:27:19.207225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:19.207225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:19.207225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:19.207225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:19.207225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:19.207225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:19.207225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:19.207225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:19.207225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:19.207225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:19.207225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:19.207225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:19.207225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:19.207225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:19.207225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:19.207225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:19.208095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:27:19.208095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:27:19.208095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:19.208095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:19.208095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 463/942] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:19.214314 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aff1d9-9391-4c48-965d-0507ed91be8a None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:19.254967 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:19.302826 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:19.323803 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc9d33-443a-43b0-a919-1b2649211468 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:19.344976 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aff1d9-9391-4c48-965d-0507ed91be8a None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:19.383292 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2a66a-fd92-4e16-b54d-58f4c0adec1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:19.384385 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8311de-bf0a-48be-ba2b-7af373f96b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:19.384674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2a66a-fd92-4e16-b54d-58f4c0adec1e None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:19.385003 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8311de-bf0a-48be-ba2b-7af373f96b8a None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:19.385284 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8311de-bf0a-48be-ba2b-7af373f96b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:19.385639 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8311de-bf0a-48be-ba2b-7af373f96b8a None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:19.385850 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2a66a-fd92-4e16-b54d-58f4c0adec1e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:19.386321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2a66a-fd92-4e16-b54d-58f4c0adec1e None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:19.387211 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2a66a-fd92-4e16-b54d-58f4c0adec1e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:19.388071 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8311de-bf0a-48be-ba2b-7af373f96b8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:19.388380 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8311de-bf0a-48be-ba2b-7af373f96b8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:19.389271 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2a66a-fd92-4e16-b54d-58f4c0adec1e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:19.389849 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8311de-bf0a-48be-ba2b-7af373f96b8a None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:19.394250 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: WARNING keystone.server.flask.common [None req-faf2a66a-fd92-4e16-b54d-58f4c0adec1e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 04 04:27:19.404002 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 464/943] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:27:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:19.409460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8311de-bf0a-48be-ba2b-7af373f96b8a None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:19.418138 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.sql.core [None req-faf2a66a-fd92-4e16-b54d-58f4c0adec1e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 04:27:19.418138 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 04 04:27:19.418138 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [parameters: {'id': 'd7005a7aadb54fe398603cd599bcacc0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 04 04:27:19.418138 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80402) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 04:27:19.426809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-faf2a66a-fd92-4e16-b54d-58f4c0adec1e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d7005a7aadb54fe398603cd599bcacc0.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d7005a7aadb54fe398603cd599bcacc0. Aug 04 04:27:19.426809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:19.426809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 04:27:19.426809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 04:27:19.426809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 04:27:19.426809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 04:27:19.426809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 04:27:19.426809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 04:27:19.426809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 04:27:19.426809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application conn.query(q) Aug 04 04:27:19.426809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 04:27:19.426809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 04:27:19.426809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 04:27:19.427658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application result.read() Aug 04 04:27:19.427658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 04:27:19.427658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 04:27:19.427658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 04:27:19.427658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 04:27:19.427658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 04:27:19.427658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 04:27:19.427658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 04:27:19.427658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 04:27:19.427658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 04:27:19.427658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:19.427658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 04:27:19.427658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:19.427658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:19.427658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 04:27:19.427658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 04:27:19.427658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 04 04:27:19.427658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 04 04:27:19.428628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 04:27:19.428628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application next(self.gen) Aug 04 04:27:19.428628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 04:27:19.428628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application yield resource Aug 04 04:27:19.428628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 04:27:19.428628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application next(self.gen) Aug 04 04:27:19.428628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 04:27:19.428628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 04:27:19.428628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 04:27:19.428628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 04:27:19.428628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 04:27:19.428628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise self.value Aug 04 04:27:19.428628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 04:27:19.428628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 04:27:19.428628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 04:27:19.428628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application session.commit() Aug 04 04:27:19.428628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 04:27:19.429517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 04:27:19.429517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 04:27:19.429517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 04:27:19.429517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 04:27:19.429517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self.session.flush() Aug 04 04:27:19.429517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 04:27:19.429517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 04:27:19.429517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 04:27:19.429517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 04:27:19.429517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 04:27:19.429517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application compat.raise_( Aug 04 04:27:19.429517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 04:27:19.429517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception Aug 04 04:27:19.429517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 04:27:19.429517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 04:27:19.429517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 04:27:19.429517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 04:27:19.431695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 04:27:19.431695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 04:27:19.431695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 04:27:19.431695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 04:27:19.431695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 04 04:27:19.431695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 04 04:27:19.431695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 04:27:19.431695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 04:27:19.431695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 04:27:19.431695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 04:27:19.431695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 04:27:19.431695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 04:27:19.431695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 04:27:19.431695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 04:27:19.431695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 04:27:19.433604 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 04:27:19.433604 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 04:27:19.433604 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception Aug 04 04:27:19.433604 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 04:27:19.433604 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 04:27:19.433604 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 04:27:19.433604 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 04:27:19.433604 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 04:27:19.433604 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 04:27:19.433604 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 04:27:19.433604 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application conn.query(q) Aug 04 04:27:19.433604 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 04:27:19.433604 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 04:27:19.433604 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 04:27:19.433604 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application result.read() Aug 04 04:27:19.433604 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 04:27:19.433604 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 04:27:19.436015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 04:27:19.436015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 04:27:19.436015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 04:27:19.436015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 04:27:19.436015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 04:27:19.436015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 04:27:19.436015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 04:27:19.436015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 04 04:27:19.436015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [parameters: {'id': 'd7005a7aadb54fe398603cd599bcacc0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 04 04:27:19.436015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 04:27:19.436015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:19.436015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:27:19.436015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:19.436015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:19.436015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 04 04:27:19.439386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 04 04:27:19.439386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 04:27:19.439386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 04:27:19.439386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 04 04:27:19.439386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:19.439386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:27:19.439386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:19.439386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:19.439386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:19.439386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:19.439386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:19.439386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:19.439386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:19.439386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:19.439386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:19.439386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:19.439386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:19.442437 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:19.442437 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 04 04:27:19.442437 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 04 04:27:19.442437 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:19.442437 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:19.442437 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 04 04:27:19.442437 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 04 04:27:19.442437 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d7005a7aadb54fe398603cd599bcacc0. Aug 04 04:27:19.442437 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:19.442437 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 480/944] 10.222.0.47 () {60 vars in 1140 bytes} [Thu Aug 4 04:27:19 2022] POST /identity/v3/projects => generated 262 bytes in 246 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:19.442437 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2a66a-fd92-4e16-b54d-58f4c0adec1e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:19.558804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:19.559537 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2a66a-fd92-4e16-b54d-58f4c0adec1e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:19.573806 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8311de-bf0a-48be-ba2b-7af373f96b8a None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:19.599658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3371eb7-23ca-46f7-b9fd-abff80875fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:19.600868 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3371eb7-23ca-46f7-b9fd-abff80875fee None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:19.601159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3371eb7-23ca-46f7-b9fd-abff80875fee None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:19.601438 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3371eb7-23ca-46f7-b9fd-abff80875fee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:19.611166 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3371eb7-23ca-46f7-b9fd-abff80875fee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:19.613529 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3371eb7-23ca-46f7-b9fd-abff80875fee None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:19.617160 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c73385-e6c3-4bb4-a8b5-228b54aeb566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:19.618080 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c73385-e6c3-4bb4-a8b5-228b54aeb566 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:19.618587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c73385-e6c3-4bb4-a8b5-228b54aeb566 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:19.619061 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c73385-e6c3-4bb4-a8b5-228b54aeb566 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:19.626823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-b3371eb7-23ca-46f7-b9fd-abff80875fee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:19.626823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:19.626823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:19.626823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:19.626823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:19.626823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:19.626823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:19.626823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:19.626823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:19.626823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:19.626823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:19.626823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:19.626823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:27:19.626823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:27:19.626823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:27:19.627709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:27:19.627709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:19.627709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:19.627709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:19.627709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:19.627709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:19.627709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:19.627709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:19.627709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:19.627709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:19.627709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:19.627709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:19.627709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:19.627709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:19.627709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:19.627709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:19.627709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:19.628571 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:27:19.628571 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:27:19.628571 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:27:19.628571 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:27:19.628571 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:27:19.628571 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:27:19.628571 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:19.628571 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:19.628571 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 481/945] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:27:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:19.629689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c73385-e6c3-4bb4-a8b5-228b54aeb566 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:19.631344 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c73385-e6c3-4bb4-a8b5-228b54aeb566 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:19.633831 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3371eb7-23ca-46f7-b9fd-abff80875fee None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:19.637660 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:19.646977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-f2c73385-e6c3-4bb4-a8b5-228b54aeb566 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:19.646977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:19.646977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:19.646977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:19.646977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:19.646977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:19.646977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:19.646977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:19.646977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:19.646977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:19.646977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:19.646977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:19.646977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:27:19.646977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:27:19.646977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:27:19.647859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:27:19.647859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:19.647859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:19.647859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:19.647859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:19.647859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:19.647859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:19.647859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:19.647859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:19.647859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:19.647859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:19.647859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:19.647859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:19.647859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:19.647859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:19.647859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:19.647859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:19.648730 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:27:19.648730 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:27:19.648730 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:27:19.648730 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:27:19.648730 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:27:19.648730 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:27:19.648730 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:19.648730 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:19.652442 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 465/946] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:27:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:19.658013 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c73385-e6c3-4bb4-a8b5-228b54aeb566 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:19.746140 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3371eb7-23ca-46f7-b9fd-abff80875fee None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:19.772828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c73385-e6c3-4bb4-a8b5-228b54aeb566 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:19.783343 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87db9a5-58b3-4313-8b0a-8a1320ddfbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:19.783974 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87db9a5-58b3-4313-8b0a-8a1320ddfbf7 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:19.784266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87db9a5-58b3-4313-8b0a-8a1320ddfbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:19.784524 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87db9a5-58b3-4313-8b0a-8a1320ddfbf7 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:19.785320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87db9a5-58b3-4313-8b0a-8a1320ddfbf7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:19.785652 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87db9a5-58b3-4313-8b0a-8a1320ddfbf7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:19.787065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87db9a5-58b3-4313-8b0a-8a1320ddfbf7 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:19.809185 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 482/947] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Aug 4 04:27:19 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:19.809506 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f8b30-41ee-400c-8823-f0ee56b75a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:19.810154 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f8b30-41ee-400c-8823-f0ee56b75a28 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:19.810436 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f8b30-41ee-400c-8823-f0ee56b75a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:19.810711 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f8b30-41ee-400c-8823-f0ee56b75a28 None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:19.811467 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f8b30-41ee-400c-8823-f0ee56b75a28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:19.811774 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f8b30-41ee-400c-8823-f0ee56b75a28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:19.812130 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3e81f-3380-43ef-bf82-8cfb9a878f06 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:19.813185 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3e81f-3380-43ef-bf82-8cfb9a878f06 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:19.813694 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3e81f-3380-43ef-bf82-8cfb9a878f06 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:19.813798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f8b30-41ee-400c-8823-f0ee56b75a28 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:19.815651 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 483/948] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:19.823124 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bc29b-2486-4380-886e-fb9a5616023b None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:19.823477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bc29b-2486-4380-886e-fb9a5616023b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:19.823693 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bc29b-2486-4380-886e-fb9a5616023b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:19.836285 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 466/949] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:27:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:19.837609 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f8b30-41ee-400c-8823-f0ee56b75a28 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:19.845926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-0b0bc29b-2486-4380-886e-fb9a5616023b None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:19.847525 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0b0bc29b-2486-4380-886e-fb9a5616023b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:19.852881 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:19.933379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:19.954248 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 484/950] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:19.980131 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f8b30-41ee-400c-8823-f0ee56b75a28 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:19.980744 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0bc29b-2486-4380-886e-fb9a5616023b None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:20.021363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb6305-a104-4981-ad39-5f5892b93d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:20.022069 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb6305-a104-4981-ad39-5f5892b93d55 None admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:20.022347 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb6305-a104-4981-ad39-5f5892b93d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:20.022618 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb6305-a104-4981-ad39-5f5892b93d55 None admin] PATH_INFO: `/v3/projects/f37a5f3140ec431da8ea14568654114a/users/3a721708c7e443e799349add27de8f84/roles/3747dec438cd4349bd9e402694b88574` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:20.025345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb6305-a104-4981-ad39-5f5892b93d55 None admin] RBAC: Authorizing `identity:create_grant(project_id=f37a5f3140ec431da8ea14568654114a, user_id=3a721708c7e443e799349add27de8f84, role_id=3747dec438cd4349bd9e402694b88574)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:20.026879 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb6305-a104-4981-ad39-5f5892b93d55 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:20.042995 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 467/951] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Aug 4 04:27:19 2022] PUT /identity/v3/projects/f37a5f3140ec431da8ea14568654114a/users/3a721708c7e443e799349add27de8f84/roles/3747dec438cd4349bd9e402694b88574 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:27:20.113953 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0bc29b-2486-4380-886e-fb9a5616023b None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:20.155186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546a5da-5a48-4270-992c-bbe848ecf2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:20.156771 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546a5da-5a48-4270-992c-bbe848ecf2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:20.157452 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546a5da-5a48-4270-992c-bbe848ecf2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:20.157893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546a5da-5a48-4270-992c-bbe848ecf2a6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:20.158765 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c546a5da-5a48-4270-992c-bbe848ecf2a6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:20.160333 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c546a5da-5a48-4270-992c-bbe848ecf2a6 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:20.170731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-c546a5da-5a48-4270-992c-bbe848ecf2a6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 04:27:20.170731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:20.170731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:27:20.170731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:27:20.170731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:27:20.170731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:27:20.170731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:27:20.170731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:20.170731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 04:27:20.170731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:20.170731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:27:20.170731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:20.170731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:20.170731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:20.170731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:20.170731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:20.170731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:20.172542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:20.172542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:20.172542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:20.172542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:20.172542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:20.172542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:20.172542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:27:20.172542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:27:20.172542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:27:20.172542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:27:20.172542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:20.172542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:20.172542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:20.172542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:20.172542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:20.172542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:20.173739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:20.173739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:20.173739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:20.173739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:20.173739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:20.173739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:20.173739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:20.173739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:20.173739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:20.173739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:20.173739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:27:20.173739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:20.173739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 04:27:20.173739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:20.177082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 485/952] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Aug 4 04:27:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:20.183642 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb6305-a104-4981-ad39-5f5892b93d55 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:20.264045 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:20.307069 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb6305-a104-4981-ad39-5f5892b93d55 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:20.347282 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18415445-64a1-4b8a-a659-371dae022ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:20.347908 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18415445-64a1-4b8a-a659-371dae022ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:20.348185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18415445-64a1-4b8a-a659-371dae022ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:20.348453 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18415445-64a1-4b8a-a659-371dae022ad1 None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:20.349137 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18415445-64a1-4b8a-a659-371dae022ad1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:20.349438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18415445-64a1-4b8a-a659-371dae022ad1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:20.350938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18415445-64a1-4b8a-a659-371dae022ad1 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:20.356426 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 468/953] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Aug 4 04:27:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:20.362045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546a5da-5a48-4270-992c-bbe848ecf2a6 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:20.424103 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:20.471250 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546a5da-5a48-4270-992c-bbe848ecf2a6 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:20.519618 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a29f3-9932-4071-8b78-98ffe54f2967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:20.520246 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a29f3-9932-4071-8b78-98ffe54f2967 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:20.520529 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a29f3-9932-4071-8b78-98ffe54f2967 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:20.520805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a29f3-9932-4071-8b78-98ffe54f2967 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:20.521497 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a29f3-9932-4071-8b78-98ffe54f2967 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:20.521830 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a29f3-9932-4071-8b78-98ffe54f2967 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:20.523204 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a29f3-9932-4071-8b78-98ffe54f2967 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:20.530438 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 486/954] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Aug 4 04:27:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:20.536779 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18415445-64a1-4b8a-a659-371dae022ad1 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:20.650924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18415445-64a1-4b8a-a659-371dae022ad1 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:20.689561 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a53bc-0507-447f-8a79-eba3bc11b423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:20.690196 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a53bc-0507-447f-8a79-eba3bc11b423 None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:20.690474 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a53bc-0507-447f-8a79-eba3bc11b423 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:20.690743 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a53bc-0507-447f-8a79-eba3bc11b423 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:20.691466 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a53bc-0507-447f-8a79-eba3bc11b423 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:20.692824 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a53bc-0507-447f-8a79-eba3bc11b423 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:20.697528 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:20.719003 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 469/955] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:27:20 2022] POST /identity/v3/endpoints => generated 349 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:27:21.217473 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74564ca4-3439-4f8a-93ec-8c0bfbaa04d2 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:21.218213 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74564ca4-3439-4f8a-93ec-8c0bfbaa04d2 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:21.218213 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74564ca4-3439-4f8a-93ec-8c0bfbaa04d2 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:21.219755 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 487/956] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:21.225132 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa11856-8a5e-4547-8c8d-3fca1d828873 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:21.226047 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa11856-8a5e-4547-8c8d-3fca1d828873 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:21.226535 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa11856-8a5e-4547-8c8d-3fca1d828873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:21.246604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-8fa11856-8a5e-4547-8c8d-3fca1d828873 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:21.247793 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-8fa11856-8a5e-4547-8c8d-3fca1d828873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:21.331829 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 470/957] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:21.353916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74564ca4-3439-4f8a-93ec-8c0bfbaa04d2 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:21.462014 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74564ca4-3439-4f8a-93ec-8c0bfbaa04d2 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:21.486184 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:21.498139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee4141-6ab2-42ad-a58f-d567c1743213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:21.498776 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee4141-6ab2-42ad-a58f-d567c1743213 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:21.499063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee4141-6ab2-42ad-a58f-d567c1743213 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:21.499335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee4141-6ab2-42ad-a58f-d567c1743213 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:21.503183 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ee4141-6ab2-42ad-a58f-d567c1743213 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:21.503183 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ee4141-6ab2-42ad-a58f-d567c1743213 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:21.512780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-70ee4141-6ab2-42ad-a58f-d567c1743213 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 04:27:21.512780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:21.512780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:27:21.512780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:27:21.512780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:27:21.512780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:27:21.512780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:27:21.512780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:21.512780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 04:27:21.512780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:21.512780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:27:21.512780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:21.512780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:21.512780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:21.512780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:21.512780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:21.512780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:21.513985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:21.513985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:21.513985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:21.513985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:21.513985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:21.513985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:21.513985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:27:21.513985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:27:21.513985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:27:21.513985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:27:21.513985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:21.513985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:21.513985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:21.513985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:21.513985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:21.513985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:21.514883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:21.514883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:21.514883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:21.514883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:21.514883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:21.514883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:21.514883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:21.514883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:21.514883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:21.514883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:21.514883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:27:21.514883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:21.514883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 04:27:21.514883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:21.521585 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 488/958] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:27:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:21.522659 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa11856-8a5e-4547-8c8d-3fca1d828873 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:21.624537 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa11856-8a5e-4547-8c8d-3fca1d828873 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:21.642160 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:21.660712 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcae12-270b-4782-8acd-00cfec972de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:21.661328 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcae12-270b-4782-8acd-00cfec972de0 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:21.661637 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcae12-270b-4782-8acd-00cfec972de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:21.661911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcae12-270b-4782-8acd-00cfec972de0 None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:21.662761 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcae12-270b-4782-8acd-00cfec972de0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:21.663065 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcae12-270b-4782-8acd-00cfec972de0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:21.664419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcae12-270b-4782-8acd-00cfec972de0 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:21.669837 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 471/959] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Aug 4 04:27:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:21.675899 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee4141-6ab2-42ad-a58f-d567c1743213 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:21.785536 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee4141-6ab2-42ad-a58f-d567c1743213 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:21.824194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e16b46-f895-4514-8fab-bb0ab3c7a147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:21.825177 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e16b46-f895-4514-8fab-bb0ab3c7a147 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:21.825729 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e16b46-f895-4514-8fab-bb0ab3c7a147 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:21.826211 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e16b46-f895-4514-8fab-bb0ab3c7a147 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:21.827116 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e16b46-f895-4514-8fab-bb0ab3c7a147 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:21.827623 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e16b46-f895-4514-8fab-bb0ab3c7a147 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:21.829330 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e16b46-f895-4514-8fab-bb0ab3c7a147 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:21.839515 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 489/960] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Aug 4 04:27:21 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:22.117645 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce050b2f-dbc7-430b-9345-1ab17a5c4f28 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:22.117949 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce050b2f-dbc7-430b-9345-1ab17a5c4f28 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:22.118249 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce050b2f-dbc7-430b-9345-1ab17a5c4f28 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:22.119860 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 472/961] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:22.125117 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5cbba-34de-4652-9b4b-1bfb94729ec9 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:22.125401 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5cbba-34de-4652-9b4b-1bfb94729ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:22.125712 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5cbba-34de-4652-9b4b-1bfb94729ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:22.144356 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-72d5cbba-34de-4652-9b4b-1bfb94729ec9 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:22.145555 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-72d5cbba-34de-4652-9b4b-1bfb94729ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:22.196304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:22.221970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 490/962] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:22.246509 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce050b2f-dbc7-430b-9345-1ab17a5c4f28 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:22.252290 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f01dd-9f38-4f3c-b9bd-e9ca6de56e38 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:22.252579 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f01dd-9f38-4f3c-b9bd-e9ca6de56e38 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:22.252860 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f01dd-9f38-4f3c-b9bd-e9ca6de56e38 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:22.254695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 491/963] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:27:22.259982 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48337227-a0db-47a0-8255-b1f0fff03a9c None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:22.260271 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48337227-a0db-47a0-8255-b1f0fff03a9c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:22.260696 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48337227-a0db-47a0-8255-b1f0fff03a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:22.280675 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-48337227-a0db-47a0-8255-b1f0fff03a9c None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:22.282680 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-48337227-a0db-47a0-8255-b1f0fff03a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:22.370012 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce050b2f-dbc7-430b-9345-1ab17a5c4f28 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:22.374020 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 492/964] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 04:27:22.403221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48337227-a0db-47a0-8255-b1f0fff03a9c None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:22.423261 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5136e47-eb4d-47ef-a8e4-8d86aad58ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:22.423907 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5136e47-eb4d-47ef-a8e4-8d86aad58ca5 None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:22.424470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5136e47-eb4d-47ef-a8e4-8d86aad58ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:22.424766 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5136e47-eb4d-47ef-a8e4-8d86aad58ca5 None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:22.425465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5136e47-eb4d-47ef-a8e4-8d86aad58ca5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:22.426987 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5136e47-eb4d-47ef-a8e4-8d86aad58ca5 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:22.430431 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: WARNING keystone.server.flask.common [None req-e5136e47-eb4d-47ef-a8e4-8d86aad58ca5 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 04 04:27:22.448143 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.sql.core [None req-e5136e47-eb4d-47ef-a8e4-8d86aad58ca5 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 04:27:22.448143 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 04 04:27:22.448143 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [parameters: {'user_id': 'fbfe67becc9247969e5198ae390e872b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 04 04:27:22.448143 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80403) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 04:27:22.470591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-e5136e47-eb4d-47ef-a8e4-8d86aad58ca5 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 04 04:27:22.470591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:22.470591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 04:27:22.470591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 04:27:22.470591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 04:27:22.470591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 04:27:22.470591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 04:27:22.470591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 04:27:22.470591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 04:27:22.470591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application conn.query(q) Aug 04 04:27:22.470591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 04:27:22.470591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 04:27:22.470591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 04:27:22.470591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application result.read() Aug 04 04:27:22.470591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 04:27:22.471552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 04:27:22.471552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 04:27:22.471552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 04:27:22.471552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 04:27:22.471552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 04:27:22.471552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 04:27:22.471552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 04:27:22.471552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 04:27:22.471552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:22.471552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 04:27:22.471552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:22.471552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:22.471552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 04:27:22.471552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 04:27:22.471552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 04 04:27:22.471552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 04 04:27:22.471552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 04:27:22.471552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application next(self.gen) Aug 04 04:27:22.472548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 04:27:22.472548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application yield resource Aug 04 04:27:22.472548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 04:27:22.472548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application next(self.gen) Aug 04 04:27:22.472548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 04:27:22.472548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 04:27:22.472548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 04:27:22.472548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 04:27:22.472548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 04:27:22.472548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise self.value Aug 04 04:27:22.472548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 04:27:22.472548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 04:27:22.472548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 04:27:22.472548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application session.commit() Aug 04 04:27:22.472548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 04:27:22.472548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 04:27:22.472548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 04:27:22.473583 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 04:27:22.473583 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 04:27:22.473583 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self.session.flush() Aug 04 04:27:22.473583 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 04:27:22.473583 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 04:27:22.473583 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 04:27:22.473583 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 04:27:22.473583 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 04:27:22.473583 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application compat.raise_( Aug 04 04:27:22.473583 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 04:27:22.473583 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception Aug 04 04:27:22.473583 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 04:27:22.473583 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 04:27:22.473583 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 04:27:22.473583 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 04:27:22.473583 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 04:27:22.473583 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 04:27:22.475469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 04:27:22.475469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 04:27:22.475469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 04 04:27:22.475469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 04 04:27:22.475469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 04:27:22.475469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 04:27:22.475469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 04:27:22.475469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 04:27:22.475469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 04:27:22.475469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 04:27:22.475469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 04:27:22.475469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 04:27:22.475469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 04:27:22.475469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 04:27:22.475469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 04:27:22.475469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception Aug 04 04:27:22.477832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 04:27:22.477832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 04:27:22.477832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 04:27:22.477832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 04:27:22.477832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 04:27:22.477832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 04:27:22.477832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 04:27:22.477832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application conn.query(q) Aug 04 04:27:22.477832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 04:27:22.477832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 04:27:22.477832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 04:27:22.477832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application result.read() Aug 04 04:27:22.477832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 04:27:22.477832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 04:27:22.477832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 04:27:22.477832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 04:27:22.479891 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 04:27:22.479891 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 04:27:22.479891 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 04:27:22.479891 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 04:27:22.479891 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 04:27:22.479891 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 04 04:27:22.479891 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [parameters: {'user_id': 'fbfe67becc9247969e5198ae390e872b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 04 04:27:22.479891 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 04:27:22.479891 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:22.479891 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:27:22.479891 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:22.479891 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:22.479891 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:22.479891 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:22.479891 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:22.481869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:22.481869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:22.481869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:22.481869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:22.481869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:22.481869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:22.481869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:22.481869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 04 04:27:22.481869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 04 04:27:22.481869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:22.481869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:22.481869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:27:22.481869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:22.481869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:27:22.481869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:22.481869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 04 04:27:22.481869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 04 04:27:22.484181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 04 04:27:22.484181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 04 04:27:22.484181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 04:27:22.484181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 04:27:22.484181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 04 04:27:22.484181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:22.484181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 473/965] 10.222.0.47 () {60 vars in 1131 bytes} [Thu Aug 4 04:27:22 2022] POST /identity/v3/users => generated 160 bytes in 226 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:22.484181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5136e47-eb4d-47ef-a8e4-8d86aad58ca5 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:22.538855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48337227-a0db-47a0-8255-b1f0fff03a9c None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:22.539497 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:22.557921 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:22.582494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca194b-4046-4778-8a8a-6db30b3de586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:22.583704 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca194b-4046-4778-8a8a-6db30b3de586 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:22.584206 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca194b-4046-4778-8a8a-6db30b3de586 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:22.584682 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca194b-4046-4778-8a8a-6db30b3de586 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:22.597302 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca194b-4046-4778-8a8a-6db30b3de586 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:27:22.603411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-79ca194b-4046-4778-8a8a-6db30b3de586 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 04:27:22.603411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:22.603411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:22.603411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:22.603411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:22.603411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:22.603411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:22.603411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:22.603411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:22.603411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:22.603411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:22.603411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:22.603411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:27:22.603411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:27:22.603411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:27:22.603411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:27:22.604347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:27:22.604347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:27:22.604347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:27:22.604347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:27:22.604347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:22.604347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:22.604347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:22.604347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:22.604347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:22.604347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:22.604347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:22.604347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:22.604347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:22.604347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:22.604347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:22.604347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:22.605168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:22.605168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:22.605168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:22.605168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:22.605168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:27:22.605168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:27:22.605168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:27:22.605168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:27:22.605168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:27:22.605168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:27:22.605168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 04:27:22.605168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:22.612725 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5136e47-eb4d-47ef-a8e4-8d86aad58ca5 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:22.613053 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 493/966] 10.222.0.47 () {56 vars in 1120 bytes} [Thu Aug 4 04:27:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:22.614836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca194b-4046-4778-8a8a-6db30b3de586 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:22.664831 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a49e3-8f1f-4da3-8fd1-2afcdf4cc053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:22.665900 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a49e3-8f1f-4da3-8fd1-2afcdf4cc053 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:22.666243 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a49e3-8f1f-4da3-8fd1-2afcdf4cc053 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:22.666526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a49e3-8f1f-4da3-8fd1-2afcdf4cc053 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:22.678133 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a49e3-8f1f-4da3-8fd1-2afcdf4cc053 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:22.679834 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a49e3-8f1f-4da3-8fd1-2afcdf4cc053 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:22.697084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-301a49e3-8f1f-4da3-8fd1-2afcdf4cc053 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 04:27:22.697084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:22.697084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:22.697084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:22.697084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:22.697084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:22.697084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:22.697084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:22.697084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:22.697084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:22.697084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:22.697084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:22.697084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:27:22.697084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:27:22.697084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:27:22.697084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:27:22.698106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:22.698106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:22.698106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:27:22.698106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:22.698106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:27:22.698106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:22.698106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:22.698106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:22.698106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:22.698106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:22.698106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:22.698106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:22.698106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:22.698106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:22.698106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:22.698106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:22.698106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:22.698106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:22.699028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:22.699028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:22.699028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:27:22.699028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:27:22.699028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:27:22.699028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:27:22.699028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:27:22.699028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:27:22.699028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 04:27:22.699028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:22.699028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 474/967] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:27:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:22.703999 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a49e3-8f1f-4da3-8fd1-2afcdf4cc053 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:22.740344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca194b-4046-4778-8a8a-6db30b3de586 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:22.782659 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3244a6-ad7e-4360-9f08-0a559701c172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:22.783538 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3244a6-ad7e-4360-9f08-0a559701c172 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:22.783830 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3244a6-ad7e-4360-9f08-0a559701c172 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:22.784109 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3244a6-ad7e-4360-9f08-0a559701c172 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:22.785270 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3244a6-ad7e-4360-9f08-0a559701c172 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:22.785613 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3244a6-ad7e-4360-9f08-0a559701c172 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:22.787284 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3244a6-ad7e-4360-9f08-0a559701c172 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:22.801592 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 494/968] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Aug 4 04:27:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:22.805990 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3244a6-ad7e-4360-9f08-0a559701c172 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:22.837858 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a49e3-8f1f-4da3-8fd1-2afcdf4cc053 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:22.839688 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:22.852275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:22.878771 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db5a85-d38f-4028-83f9-2d7360c01c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:22.879592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db5a85-d38f-4028-83f9-2d7360c01c53 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:22.879954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db5a85-d38f-4028-83f9-2d7360c01c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:22.880393 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db5a85-d38f-4028-83f9-2d7360c01c53 None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:22.881218 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1db5a85-d38f-4028-83f9-2d7360c01c53 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:22.881527 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1db5a85-d38f-4028-83f9-2d7360c01c53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:22.883113 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1db5a85-d38f-4028-83f9-2d7360c01c53 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:22.925998 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3244a6-ad7e-4360-9f08-0a559701c172 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:22.934235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 475/969] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Aug 4 04:27:22 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:22.934235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a53b9-3610-4a0d-aaf9-c8eb5278ec12 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:22.934235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a53b9-3610-4a0d-aaf9-c8eb5278ec12 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:22.934235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a53b9-3610-4a0d-aaf9-c8eb5278ec12 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:22.938162 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 476/970] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:22.939914 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdfb96-b252-496f-8413-c5ee14e43059 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:22.940052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdfb96-b252-496f-8413-c5ee14e43059 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:22.940445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdfb96-b252-496f-8413-c5ee14e43059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:22.961328 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-dbbdfb96-b252-496f-8413-c5ee14e43059 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:22.963223 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-dbbdfb96-b252-496f-8413-c5ee14e43059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:22.970405 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f11ae-c94f-49b2-a953-3f5b1fccad9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:22.971361 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f11ae-c94f-49b2-a953-3f5b1fccad9d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:22.971824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f11ae-c94f-49b2-a953-3f5b1fccad9d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:22.972290 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f11ae-c94f-49b2-a953-3f5b1fccad9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:22.985582 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f11ae-c94f-49b2-a953-3f5b1fccad9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:22.987074 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f11ae-c94f-49b2-a953-3f5b1fccad9d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:23.004043 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-d79f11ae-c94f-49b2-a953-3f5b1fccad9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:23.004043 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:23.004043 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:23.004043 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:23.004043 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:23.004043 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:23.004043 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:23.004043 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:23.004043 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:23.004043 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:23.004043 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:23.004043 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:23.004043 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:27:23.004043 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:27:23.004043 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:27:23.005186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:27:23.005186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:23.005186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:23.005186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:23.005186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:23.005186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:23.005186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:23.005186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:23.005186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:23.005186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:23.005186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:23.005186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:23.005186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:23.005186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:23.005186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:23.005186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:23.005186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:23.006265 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:27:23.006265 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:27:23.006265 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:23.006265 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:23.006265 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 495/971] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:23.010939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f11ae-c94f-49b2-a953-3f5b1fccad9d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:23.080357 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 477/972] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:23.106214 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbdfb96-b252-496f-8413-c5ee14e43059 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:23.130457 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:23.150911 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f11ae-c94f-49b2-a953-3f5b1fccad9d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:23.192533 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f23b6c-78ad-403c-814b-fda1261c1eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:23.193583 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f23b6c-78ad-403c-814b-fda1261c1eaa None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:23.193583 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f23b6c-78ad-403c-814b-fda1261c1eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:23.194811 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f23b6c-78ad-403c-814b-fda1261c1eaa None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:23.194811 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f23b6c-78ad-403c-814b-fda1261c1eaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:23.195149 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f23b6c-78ad-403c-814b-fda1261c1eaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:23.196587 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f23b6c-78ad-403c-814b-fda1261c1eaa None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:23.211583 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 496/973] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:23.217645 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f23b6c-78ad-403c-814b-fda1261c1eaa None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:23.229503 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:23.240623 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbdfb96-b252-496f-8413-c5ee14e43059 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:23.281385 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5528cd7-24c7-44d7-a49a-600c282c18cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:23.282693 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5528cd7-24c7-44d7-a49a-600c282c18cc None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:23.282980 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5528cd7-24c7-44d7-a49a-600c282c18cc None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:23.283392 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5528cd7-24c7-44d7-a49a-600c282c18cc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:23.284168 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5528cd7-24c7-44d7-a49a-600c282c18cc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:23.286338 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5528cd7-24c7-44d7-a49a-600c282c18cc None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:23.299291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-f5528cd7-24c7-44d7-a49a-600c282c18cc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 04:27:23.299291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:23.299291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:27:23.299291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:27:23.299291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:27:23.299291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:27:23.299291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:27:23.299291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:23.299291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 04:27:23.299291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:23.299291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:27:23.299291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:23.299291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:23.299291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:23.299291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:23.299291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:23.299291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:23.300383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:23.300383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:23.300383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:23.300383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:23.300383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:23.300383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:23.300383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:27:23.300383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:27:23.300383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:27:23.300383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:27:23.300383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:23.300383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:23.300383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:23.300383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:23.300383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:23.300383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:23.301212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:23.301212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:23.301212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:23.301212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:23.301212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:23.301212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:23.301212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:23.301212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:23.301212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:23.301212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:23.301212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:27:23.301212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:23.301212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 04:27:23.301212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:23.301212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 478/974] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Aug 4 04:27:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:23.306444 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5528cd7-24c7-44d7-a49a-600c282c18cc None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:23.334786 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f23b6c-78ad-403c-814b-fda1261c1eaa None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:23.372285 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108e27a-10b1-4047-8624-c0be1613fdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:23.372942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108e27a-10b1-4047-8624-c0be1613fdce None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:23.373226 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108e27a-10b1-4047-8624-c0be1613fdce None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:23.373502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108e27a-10b1-4047-8624-c0be1613fdce None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:23.382032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9108e27a-10b1-4047-8624-c0be1613fdce None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:23.383625 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9108e27a-10b1-4047-8624-c0be1613fdce None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:23.396633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-9108e27a-10b1-4047-8624-c0be1613fdce None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 04:27:23.396633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:23.396633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:23.396633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:23.396633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:23.396633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:23.396633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:23.396633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:23.396633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:23.396633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:23.396633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:23.396633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:23.396633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:27:23.396633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:27:23.396633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:27:23.396633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:27:23.397835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:23.397835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:23.397835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:27:23.397835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:23.397835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:27:23.397835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:23.397835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:23.397835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:23.397835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:23.397835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:23.397835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:23.397835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:23.397835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:23.397835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:23.397835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:23.397835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:23.397835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:23.397835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:23.398800 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:23.398800 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:23.398800 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:27:23.398800 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:27:23.398800 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:27:23.398800 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:27:23.398800 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:27:23.398800 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:27:23.398800 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 04:27:23.398800 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:23.398800 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 497/975] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Aug 4 04:27:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:23.403604 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108e27a-10b1-4047-8624-c0be1613fdce None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:23.407891 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:23.409924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5528cd7-24c7-44d7-a49a-600c282c18cc None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:23.448561 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb8fad-93cd-4a68-bd74-7b414fa8063c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:23.449569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb8fad-93cd-4a68-bd74-7b414fa8063c None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:23.450068 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb8fad-93cd-4a68-bd74-7b414fa8063c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:23.450550 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb8fad-93cd-4a68-bd74-7b414fa8063c None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:23.451373 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb8fad-93cd-4a68-bd74-7b414fa8063c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:23.451833 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb8fad-93cd-4a68-bd74-7b414fa8063c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:23.454080 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb8fad-93cd-4a68-bd74-7b414fa8063c None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:23.460444 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 479/976] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Aug 4 04:27:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:23.465997 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb8fad-93cd-4a68-bd74-7b414fa8063c None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:23.490354 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:23.519999 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108e27a-10b1-4047-8624-c0be1613fdce None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:23.559197 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539c9b7-ce03-493c-abc7-6990f4d3c6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:23.560486 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539c9b7-ce03-493c-abc7-6990f4d3c6bc None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:23.560958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539c9b7-ce03-493c-abc7-6990f4d3c6bc None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:23.561413 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539c9b7-ce03-493c-abc7-6990f4d3c6bc None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:23.563128 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539c9b7-ce03-493c-abc7-6990f4d3c6bc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:23.563657 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539c9b7-ce03-493c-abc7-6990f4d3c6bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:23.565416 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539c9b7-ce03-493c-abc7-6990f4d3c6bc None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:23.584300 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb8fad-93cd-4a68-bd74-7b414fa8063c None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:23.584483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 498/977] 10.222.0.47 () {56 vars in 1149 bytes} [Thu Aug 4 04:27:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:23.589884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539c9b7-ce03-493c-abc7-6990f4d3c6bc None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:23.626688 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f28f7-5e1f-4bd9-959b-346c10b8381f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:23.627313 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f28f7-5e1f-4bd9-959b-346c10b8381f None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:23.627588 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f28f7-5e1f-4bd9-959b-346c10b8381f None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:23.627857 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f28f7-5e1f-4bd9-959b-346c10b8381f None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:23.628538 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f28f7-5e1f-4bd9-959b-346c10b8381f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:23.628842 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f28f7-5e1f-4bd9-959b-346c10b8381f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:23.630216 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f28f7-5e1f-4bd9-959b-346c10b8381f None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:23.639883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 480/978] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Aug 4 04:27:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:23.641502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f28f7-5e1f-4bd9-959b-346c10b8381f None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:23.663199 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:23.706282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539c9b7-ce03-493c-abc7-6990f4d3c6bc None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:23.732987 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:23.743640 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084dfa7-2a23-45c8-a60b-58fdb67db85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:23.744565 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084dfa7-2a23-45c8-a60b-58fdb67db85b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:23.744845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084dfa7-2a23-45c8-a60b-58fdb67db85b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:23.745278 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084dfa7-2a23-45c8-a60b-58fdb67db85b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:23.753031 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f28f7-5e1f-4bd9-959b-346c10b8381f None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:23.788361 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacac86f-2a4e-4d0d-8f0c-6cb4dbcd9ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:23.788978 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacac86f-2a4e-4d0d-8f0c-6cb4dbcd9ead None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:23.789259 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacac86f-2a4e-4d0d-8f0c-6cb4dbcd9ead None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:23.789840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacac86f-2a4e-4d0d-8f0c-6cb4dbcd9ead None admin] PATH_INFO: `/v3/endpoints` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:23.790339 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacac86f-2a4e-4d0d-8f0c-6cb4dbcd9ead None admin] RBAC: Adding query filter params (interface=admin, service_id=908a24a25ee3442d8e6980c95454aeaa, region_id=RegionOne) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:23.790646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacac86f-2a4e-4d0d-8f0c-6cb4dbcd9ead None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:23.791991 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacac86f-2a4e-4d0d-8f0c-6cb4dbcd9ead None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:23.798142 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 481/979] 10.222.0.47 () {56 vars in 1249 bytes} [Thu Aug 4 04:27:23 2022] GET /identity/v3/endpoints?service_id=908a24a25ee3442d8e6980c95454aeaa&interface=admin®ion_id=RegionOne => generated 193 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:23.803641 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacac86f-2a4e-4d0d-8f0c-6cb4dbcd9ead None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:23.821068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b084dfa7-2a23-45c8-a60b-58fdb67db85b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:23.822793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b084dfa7-2a23-45c8-a60b-58fdb67db85b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:23.887512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:23.926440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacac86f-2a4e-4d0d-8f0c-6cb4dbcd9ead None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:23.933594 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 499/980] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:27:23 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 04:27:23.939424 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084dfa7-2a23-45c8-a60b-58fdb67db85b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:23.965265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e00427c-4591-4794-94e5-5b4ee2e1e1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:23.965936 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00427c-4591-4794-94e5-5b4ee2e1e1f5 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:23.966235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00427c-4591-4794-94e5-5b4ee2e1e1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:23.966578 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00427c-4591-4794-94e5-5b4ee2e1e1f5 None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:23.967302 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e00427c-4591-4794-94e5-5b4ee2e1e1f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:23.969711 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e00427c-4591-4794-94e5-5b4ee2e1e1f5 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:23.976822 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 482/981] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Aug 4 04:27:23 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:27:23.978012 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84296f-332e-4e4c-898c-9ced2ae55d20 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:23.978368 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84296f-332e-4e4c-898c-9ced2ae55d20 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:23.978713 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84296f-332e-4e4c-898c-9ced2ae55d20 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:23.980160 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 483/982] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:23.986122 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7760485-40c3-4eaf-9e25-e64e2ea51fc4 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:23.986411 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7760485-40c3-4eaf-9e25-e64e2ea51fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:23.986692 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7760485-40c3-4eaf-9e25-e64e2ea51fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:24.007065 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-c7760485-40c3-4eaf-9e25-e64e2ea51fc4 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:24.008590 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c7760485-40c3-4eaf-9e25-e64e2ea51fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:24.013228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:24.052651 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084dfa7-2a23-45c8-a60b-58fdb67db85b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:24.089809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72336795-1d4c-4a19-a6e2-313c33d43d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:24.090466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72336795-1d4c-4a19-a6e2-313c33d43d58 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:24.090793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72336795-1d4c-4a19-a6e2-313c33d43d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:24.091010 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72336795-1d4c-4a19-a6e2-313c33d43d58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:24.093703 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 484/983] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 04:27:24.102580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72336795-1d4c-4a19-a6e2-313c33d43d58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:24.104054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72336795-1d4c-4a19-a6e2-313c33d43d58 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:24.116050 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7760485-40c3-4eaf-9e25-e64e2ea51fc4 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:24.120338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-72336795-1d4c-4a19-a6e2-313c33d43d58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:24.120338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:24.120338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:24.120338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:24.120338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:24.120338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:24.120338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:24.120338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:24.120338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:24.120338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:24.120338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:24.120338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:24.120338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:27:24.120338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:27:24.120338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:27:24.121252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:27:24.121252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:24.121252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:24.121252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:24.121252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:24.121252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:24.121252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:24.121252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:24.121252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:24.121252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:24.121252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:24.121252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:24.121252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:24.121252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:24.121252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:24.121252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:24.121252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:24.126373 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:27:24.126373 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:27:24.126373 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:24.126373 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:24.126373 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 500/984] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:24.130534 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72336795-1d4c-4a19-a6e2-313c33d43d58 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:24.152376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:24.240827 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72336795-1d4c-4a19-a6e2-313c33d43d58 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:24.243799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7760485-40c3-4eaf-9e25-e64e2ea51fc4 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:24.277260 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142e451-d354-4aa4-8648-88e206bf4cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:24.277909 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142e451-d354-4aa4-8648-88e206bf4cea None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:24.278219 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142e451-d354-4aa4-8648-88e206bf4cea None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:24.278465 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142e451-d354-4aa4-8648-88e206bf4cea None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:24.279234 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1142e451-d354-4aa4-8648-88e206bf4cea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:24.279541 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1142e451-d354-4aa4-8648-88e206bf4cea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:24.280161 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659c02e-75b2-4c30-82bd-9c11d2745f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:24.280940 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659c02e-75b2-4c30-82bd-9c11d2745f7b None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:24.281141 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1142e451-d354-4aa4-8648-88e206bf4cea None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:24.281224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659c02e-75b2-4c30-82bd-9c11d2745f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:24.281405 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659c02e-75b2-4c30-82bd-9c11d2745f7b None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:24.282122 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659c02e-75b2-4c30-82bd-9c11d2745f7b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:24.283594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659c02e-75b2-4c30-82bd-9c11d2745f7b None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:24.294752 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 501/985] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:24.295084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:24.303004 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142e451-d354-4aa4-8648-88e206bf4cea None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:24.309603 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 485/986] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Aug 4 04:27:24 2022] POST /identity/v3/services => generated 242 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 04:27:24.394471 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:24.424996 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142e451-d354-4aa4-8648-88e206bf4cea None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:24.472889 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917549b5-6e17-4cc6-a6a5-bf5dc5ee3fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:24.473655 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917549b5-6e17-4cc6-a6a5-bf5dc5ee3fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:24.474002 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917549b5-6e17-4cc6-a6a5-bf5dc5ee3fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:24.474393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917549b5-6e17-4cc6-a6a5-bf5dc5ee3fc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:24.481806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917549b5-6e17-4cc6-a6a5-bf5dc5ee3fc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:24.483328 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917549b5-6e17-4cc6-a6a5-bf5dc5ee3fc7 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:24.496239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-917549b5-6e17-4cc6-a6a5-bf5dc5ee3fc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:24.496239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:24.496239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:24.496239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:24.496239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:24.496239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:24.496239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:24.496239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:24.496239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:24.496239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:24.496239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:24.496239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:24.496239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:27:24.496239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:27:24.496239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:27:24.497329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:27:24.497329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:24.497329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:24.497329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:24.497329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:24.497329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:24.497329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:24.497329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:24.497329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:24.497329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:24.497329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:24.497329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:24.497329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:24.497329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:24.497329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:24.497329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:24.497329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:24.498452 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:27:24.498452 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:27:24.498452 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:27:24.498452 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:27:24.498452 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:27:24.498452 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:27:24.498452 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:24.498452 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:24.502904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 502/987] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:27:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:24.507590 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659c02e-75b2-4c30-82bd-9c11d2745f7b None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:24.618731 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659c02e-75b2-4c30-82bd-9c11d2745f7b None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:24.652886 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a0a84-9255-4e14-afeb-7e4a1fc283f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:24.653516 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a0a84-9255-4e14-afeb-7e4a1fc283f5 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:24.653825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a0a84-9255-4e14-afeb-7e4a1fc283f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:24.654101 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a0a84-9255-4e14-afeb-7e4a1fc283f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:24.654851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a0a84-9255-4e14-afeb-7e4a1fc283f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:24.655157 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a0a84-9255-4e14-afeb-7e4a1fc283f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:24.657580 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a0a84-9255-4e14-afeb-7e4a1fc283f5 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:24.670844 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 486/988] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:27:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:24.677205 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917549b5-6e17-4cc6-a6a5-bf5dc5ee3fc7 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:24.782054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917549b5-6e17-4cc6-a6a5-bf5dc5ee3fc7 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:24.811068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:24.818147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa34f75-9a53-42af-9cd6-d28fa217b11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:24.818840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa34f75-9a53-42af-9cd6-d28fa217b11f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:24.819197 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa34f75-9a53-42af-9cd6-d28fa217b11f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:24.819544 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa34f75-9a53-42af-9cd6-d28fa217b11f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:24.820407 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa34f75-9a53-42af-9cd6-d28fa217b11f None admin] RBAC: Adding query filter params (role.id=3747dec438cd4349bd9e402694b88574, scope.project.id=f37a5f3140ec431da8ea14568654114a, user.id=3a721708c7e443e799349add27de8f84) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:24.820792 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa34f75-9a53-42af-9cd6-d28fa217b11f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:24.822311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa34f75-9a53-42af-9cd6-d28fa217b11f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:24.828415 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 503/989] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:27:24 2022] GET /identity/v3/role_assignments?user.id=3a721708c7e443e799349add27de8f84&scope.project.id=f37a5f3140ec431da8ea14568654114a&role.id=3747dec438cd4349bd9e402694b88574 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:26.014799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acd2f2-6e3f-442e-935d-4adc030fb518 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:26.015348 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acd2f2-6e3f-442e-935d-4adc030fb518 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:26.015630 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acd2f2-6e3f-442e-935d-4adc030fb518 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:26.017151 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 487/990] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:26.023074 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104777db-8611-4342-a8c5-a2d334d26113 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:26.023611 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104777db-8611-4342-a8c5-a2d334d26113 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:26.023751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104777db-8611-4342-a8c5-a2d334d26113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:26.043749 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-104777db-8611-4342-a8c5-a2d334d26113 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:26.045186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-104777db-8611-4342-a8c5-a2d334d26113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:26.116329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 504/991] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:26 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:26.137800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3acd2f2-6e3f-442e-935d-4adc030fb518 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:26.198501 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:26.242317 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3acd2f2-6e3f-442e-935d-4adc030fb518 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:26.275954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d94e8b-ef04-4bd9-97fc-2958e48cd2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:26.276508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d94e8b-ef04-4bd9-97fc-2958e48cd2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:26.276783 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d94e8b-ef04-4bd9-97fc-2958e48cd2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:26.277050 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d94e8b-ef04-4bd9-97fc-2958e48cd2c1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:26.277698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d94e8b-ef04-4bd9-97fc-2958e48cd2c1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:26.279335 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d94e8b-ef04-4bd9-97fc-2958e48cd2c1 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:26.289265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-b4d94e8b-ef04-4bd9-97fc-2958e48cd2c1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 04:27:26.289265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:26.289265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:27:26.289265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:27:26.289265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:27:26.289265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:27:26.289265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:27:26.289265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:26.289265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 04:27:26.289265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:26.289265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:27:26.289265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:26.289265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:26.289265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:26.289265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:26.289265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:26.289265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:26.290322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:26.290322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:26.290322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:26.290322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:26.290322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:26.290322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:26.290322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:27:26.290322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:27:26.290322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:27:26.290322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:27:26.290322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:26.290322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:26.290322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:26.290322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:26.290322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:26.290322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:26.291152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:26.291152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:26.291152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:26.291152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:26.291152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:26.291152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:26.291152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:26.291152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:26.291152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:26.291152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:26.291152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:27:26.291152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:26.291152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 04:27:26.291152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:26.292531 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 488/992] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Aug 4 04:27:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:26.297532 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104777db-8611-4342-a8c5-a2d334d26113 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:26.378731 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb0b57-4e87-477a-89cf-f5f245bec370 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:26.379086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb0b57-4e87-477a-89cf-f5f245bec370 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:26.379289 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb0b57-4e87-477a-89cf-f5f245bec370 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:26.380634 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 489/993] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:27:26.385501 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1693d6-5e5d-453f-92a0-3c6c41e120b5 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:26.385831 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1693d6-5e5d-453f-92a0-3c6c41e120b5 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:26.386109 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1693d6-5e5d-453f-92a0-3c6c41e120b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:26.404644 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-9a1693d6-5e5d-453f-92a0-3c6c41e120b5 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:26.405894 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-9a1693d6-5e5d-453f-92a0-3c6c41e120b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:26.407451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104777db-8611-4342-a8c5-a2d334d26113 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:26.435299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:26.448171 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e68707-d788-4884-a20a-78d382ec6537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:26.448792 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e68707-d788-4884-a20a-78d382ec6537 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:26.449072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e68707-d788-4884-a20a-78d382ec6537 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:26.449721 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e68707-d788-4884-a20a-78d382ec6537 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:26.450050 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e68707-d788-4884-a20a-78d382ec6537 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:26.450354 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e68707-d788-4884-a20a-78d382ec6537 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:26.451721 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e68707-d788-4884-a20a-78d382ec6537 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:26.458400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 505/994] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Aug 4 04:27:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:26.463925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e68707-d788-4884-a20a-78d382ec6537 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:26.479735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 490/995] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:26 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 04:27:26.503214 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1693d6-5e5d-453f-92a0-3c6c41e120b5 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:26.565391 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:26.583163 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e68707-d788-4884-a20a-78d382ec6537 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:26.617286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1693d6-5e5d-453f-92a0-3c6c41e120b5 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:26.619584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee22f8b-dc09-4561-a7f4-c62987977ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:26.620211 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee22f8b-dc09-4561-a7f4-c62987977ead None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:26.620490 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee22f8b-dc09-4561-a7f4-c62987977ead None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:26.620766 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee22f8b-dc09-4561-a7f4-c62987977ead None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:26.621443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee22f8b-dc09-4561-a7f4-c62987977ead None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:26.621774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee22f8b-dc09-4561-a7f4-c62987977ead None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:26.623136 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee22f8b-dc09-4561-a7f4-c62987977ead None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:26.628311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 506/996] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Aug 4 04:27:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:26.634634 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee22f8b-dc09-4561-a7f4-c62987977ead None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:26.662789 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0146000d-ddb3-417c-b43b-7e931e3cd101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:26.663678 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0146000d-ddb3-417c-b43b-7e931e3cd101 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:26.663954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0146000d-ddb3-417c-b43b-7e931e3cd101 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:26.664222 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0146000d-ddb3-417c-b43b-7e931e3cd101 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:26.664841 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0146000d-ddb3-417c-b43b-7e931e3cd101 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:26.666222 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0146000d-ddb3-417c-b43b-7e931e3cd101 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:26.679245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-0146000d-ddb3-417c-b43b-7e931e3cd101 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 04:27:26.679245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:26.679245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:27:26.679245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:27:26.679245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:27:26.679245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:27:26.679245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:27:26.679245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:26.679245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 04:27:26.679245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:26.679245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:27:26.679245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:26.679245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:26.679245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:26.679245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:26.679245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:26.679245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:26.680272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:26.680272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:26.680272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:26.680272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:26.680272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:26.680272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:26.680272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:27:26.680272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:27:26.680272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:27:26.680272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:27:26.680272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:26.680272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:26.680272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:26.680272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:26.680272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:26.680272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:26.681810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:26.681810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:26.681810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:26.681810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:26.681810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:26.681810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:26.681810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:26.681810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:26.681810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:26.681810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:26.681810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:27:26.681810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:26.681810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 04:27:26.681810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:26.681810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 491/997] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:27:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:26.686371 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0146000d-ddb3-417c-b43b-7e931e3cd101 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:26.696650 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:26.745605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee22f8b-dc09-4561-a7f4-c62987977ead None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:26.785840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e4069-ba43-4538-953a-d37ab1a17be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:26.786772 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e4069-ba43-4538-953a-d37ab1a17be7 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:26.787057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e4069-ba43-4538-953a-d37ab1a17be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:26.787334 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e4069-ba43-4538-953a-d37ab1a17be7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:26.788051 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e4069-ba43-4538-953a-d37ab1a17be7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:26.789589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e4069-ba43-4538-953a-d37ab1a17be7 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:26.807225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0146000d-ddb3-417c-b43b-7e931e3cd101 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:26.817661 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 507/998] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:27:26 2022] POST /identity/v3/endpoints => generated 325 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 04:27:26.836465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:26.865303 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e5e18-b364-44a3-a593-8b4c5bc8ad0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:26.865960 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e5e18-b364-44a3-a593-8b4c5bc8ad0c None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:26.866238 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e5e18-b364-44a3-a593-8b4c5bc8ad0c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:26.866508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e5e18-b364-44a3-a593-8b4c5bc8ad0c None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:26.867199 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e5e18-b364-44a3-a593-8b4c5bc8ad0c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:26.867503 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e5e18-b364-44a3-a593-8b4c5bc8ad0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:26.868882 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e5e18-b364-44a3-a593-8b4c5bc8ad0c None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:26.875267 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 492/999] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Aug 4 04:27:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:26.880681 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e4069-ba43-4538-953a-d37ab1a17be7 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:26.890518 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1bffb-77db-4ac9-80ef-87e359932845 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:26.890845 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1bffb-77db-4ac9-80ef-87e359932845 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:26.891082 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1bffb-77db-4ac9-80ef-87e359932845 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:26.892498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 493/1000] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:27:26.898836 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97008e-ebd7-4b01-a439-510407c4af7e None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:26.899231 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97008e-ebd7-4b01-a439-510407c4af7e None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:26.899512 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97008e-ebd7-4b01-a439-510407c4af7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:26.918995 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-aa97008e-ebd7-4b01-a439-510407c4af7e None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:26.920486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-aa97008e-ebd7-4b01-a439-510407c4af7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:27.005568 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 494/1001] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:26 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 04:27:27.006739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e4069-ba43-4538-953a-d37ab1a17be7 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:27.020415 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:27.026294 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97008e-ebd7-4b01-a439-510407c4af7e None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:27.044993 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdedd31-b53b-41fb-8bbd-454040cc98a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:27.045650 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdedd31-b53b-41fb-8bbd-454040cc98a0 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:27.045929 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdedd31-b53b-41fb-8bbd-454040cc98a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:27.046814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdedd31-b53b-41fb-8bbd-454040cc98a0 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:27.047615 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdedd31-b53b-41fb-8bbd-454040cc98a0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:27.047921 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdedd31-b53b-41fb-8bbd-454040cc98a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:27.049286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdedd31-b53b-41fb-8bbd-454040cc98a0 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:27.054093 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 508/1002] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Aug 4 04:27:26 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:27.059644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdedd31-b53b-41fb-8bbd-454040cc98a0 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:27.144041 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97008e-ebd7-4b01-a439-510407c4af7e None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:27.154871 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:27.166372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdedd31-b53b-41fb-8bbd-454040cc98a0 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:27.182904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ecaff7-63b5-4d11-8513-930ef9c134a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:27.183541 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecaff7-63b5-4d11-8513-930ef9c134a7 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:27.183819 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecaff7-63b5-4d11-8513-930ef9c134a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:27.184087 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecaff7-63b5-4d11-8513-930ef9c134a7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:27.184709 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ecaff7-63b5-4d11-8513-930ef9c134a7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:27.186088 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ecaff7-63b5-4d11-8513-930ef9c134a7 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:27.195491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-56ecaff7-63b5-4d11-8513-930ef9c134a7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 04:27:27.195491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:27.195491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:27:27.195491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:27:27.195491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:27:27.195491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:27:27.195491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:27:27.195491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:27.195491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 04:27:27.195491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:27.195491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:27:27.195491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:27.195491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:27.195491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:27.195491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:27.195491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:27.195491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:27.196684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:27.196684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:27.196684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:27.196684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:27.196684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:27.196684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:27.196684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:27:27.196684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:27:27.196684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:27:27.196684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:27:27.196684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:27.196684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:27.196684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:27.196684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:27.196684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:27.196684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:27.197508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:27.197508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:27.197508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:27.197508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:27.197508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:27.197508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:27.197508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:27.197508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:27.197508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:27.197508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:27.197508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:27:27.197508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:27.197508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 04:27:27.197508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:27.197508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 495/1003] 10.222.0.47 () {56 vars in 1105 bytes} [Thu Aug 4 04:27:27 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:27.203284 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ecaff7-63b5-4d11-8513-930ef9c134a7 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:27.209364 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb1fc7-7c67-4f98-8efc-986f1e0922ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:27.210018 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb1fc7-7c67-4f98-8efc-986f1e0922ed None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:27.210335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb1fc7-7c67-4f98-8efc-986f1e0922ed None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:27.210553 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb1fc7-7c67-4f98-8efc-986f1e0922ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:27.211309 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb1fc7-7c67-4f98-8efc-986f1e0922ed None admin] RBAC: Adding query filter params (interface=public, service_id=4ed89f97aea048e78140b25b29ef8437, region_id=RegionOne) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:27.211616 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb1fc7-7c67-4f98-8efc-986f1e0922ed None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:27.212969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb1fc7-7c67-4f98-8efc-986f1e0922ed None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:27.218578 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 509/1004] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Aug 4 04:27:27 2022] GET /identity/v3/endpoints?service_id=4ed89f97aea048e78140b25b29ef8437&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:27.223977 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb1fc7-7c67-4f98-8efc-986f1e0922ed None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:27.269750 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:27.314783 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ecaff7-63b5-4d11-8513-930ef9c134a7 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:27.337151 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb1fc7-7c67-4f98-8efc-986f1e0922ed None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:27.352658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5842db2-237b-403f-9c52-f47d043b49dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:27.353301 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5842db2-237b-403f-9c52-f47d043b49dc None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:27.353598 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5842db2-237b-403f-9c52-f47d043b49dc None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:27.353871 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5842db2-237b-403f-9c52-f47d043b49dc None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:27.354554 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5842db2-237b-403f-9c52-f47d043b49dc None admin] RBAC: Adding query filter params (name=image) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:27.354889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5842db2-237b-403f-9c52-f47d043b49dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:27.356259 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5842db2-237b-403f-9c52-f47d043b49dc None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:27.362000 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 496/1005] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:27.367448 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5842db2-237b-403f-9c52-f47d043b49dc None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:27.381005 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee86b3-218c-499f-aab3-7fb3462ce7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:27.381650 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee86b3-218c-499f-aab3-7fb3462ce7f8 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:27.381946 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee86b3-218c-499f-aab3-7fb3462ce7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:27.382193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee86b3-218c-499f-aab3-7fb3462ce7f8 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:27.382843 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee86b3-218c-499f-aab3-7fb3462ce7f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:27.384427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee86b3-218c-499f-aab3-7fb3462ce7f8 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:27.390943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 510/1006] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Aug 4 04:27:27 2022] GET /identity/v3/services => generated 1727 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 04:27:27.414153 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:27.490106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5842db2-237b-403f-9c52-f47d043b49dc None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:27.536845 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ed96f-ea57-4d7d-a3b8-9517df8db3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:27.537490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ed96f-ea57-4d7d-a3b8-9517df8db3fb None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:27.537797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ed96f-ea57-4d7d-a3b8-9517df8db3fb None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:27.538066 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ed96f-ea57-4d7d-a3b8-9517df8db3fb None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:27.538748 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ed96f-ea57-4d7d-a3b8-9517df8db3fb None admin] RBAC: Adding query filter params (type=image) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:27.539049 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ed96f-ea57-4d7d-a3b8-9517df8db3fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:27.540411 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ed96f-ea57-4d7d-a3b8-9517df8db3fb None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:27.546235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 497/1007] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:27 2022] GET /identity/v3/services?type=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:28.881145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14ce13-3c21-4d16-99ba-89a467929583 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:28.881703 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14ce13-3c21-4d16-99ba-89a467929583 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:28.881818 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14ce13-3c21-4d16-99ba-89a467929583 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:28.883220 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 511/1008] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:28.888399 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fbf9e-1805-4f4c-8cbf-56314e16c13d None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:28.888823 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fbf9e-1805-4f4c-8cbf-56314e16c13d None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:28.889021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fbf9e-1805-4f4c-8cbf-56314e16c13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:28.906864 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-5b5fbf9e-1805-4f4c-8cbf-56314e16c13d None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:28.908025 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5b5fbf9e-1805-4f4c-8cbf-56314e16c13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:28.977588 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 498/1009] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:28 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:29.001589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14ce13-3c21-4d16-99ba-89a467929583 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:29.107155 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14ce13-3c21-4d16-99ba-89a467929583 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:29.110517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:29.145137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a4e52-41cd-4188-acbb-d8b6f464aa91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:29.145745 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a4e52-41cd-4188-acbb-d8b6f464aa91 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:29.146027 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a4e52-41cd-4188-acbb-d8b6f464aa91 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:29.146297 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a4e52-41cd-4188-acbb-d8b6f464aa91 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:29.147455 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a4e52-41cd-4188-acbb-d8b6f464aa91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:29.148891 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a4e52-41cd-4188-acbb-d8b6f464aa91 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:29.155181 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 512/1010] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:29.156921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fbf9e-1805-4f4c-8cbf-56314e16c13d None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:29.188775 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:29.262057 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fbf9e-1805-4f4c-8cbf-56314e16c13d None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:29.298007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b1303a-8ef3-4751-9061-896377db94fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:29.298619 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1303a-8ef3-4751-9061-896377db94fe None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:29.298894 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1303a-8ef3-4751-9061-896377db94fe None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:29.299183 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1303a-8ef3-4751-9061-896377db94fe None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:29.299890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b1303a-8ef3-4751-9061-896377db94fe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:29.301305 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b1303a-8ef3-4751-9061-896377db94fe None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:29.328896 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 499/1011] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Aug 4 04:27:29 2022] POST /identity/v3/projects => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:27:29.473675 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4099a92-10a3-4b92-a907-8720ec364974 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:29.473850 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4099a92-10a3-4b92-a907-8720ec364974 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:29.474240 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4099a92-10a3-4b92-a907-8720ec364974 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:29.476042 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 513/1012] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:29.482787 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c183e81-fef0-4d50-b183-39506952fe26 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:29.483065 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c183e81-fef0-4d50-b183-39506952fe26 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:29.484610 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c183e81-fef0-4d50-b183-39506952fe26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:29.516081 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-7c183e81-fef0-4d50-b183-39506952fe26 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:29.517301 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-7c183e81-fef0-4d50-b183-39506952fe26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:29.579609 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e26ba9-9fd8-4014-b239-9638f3aa9571 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:29.579898 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e26ba9-9fd8-4014-b239-9638f3aa9571 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:29.580179 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e26ba9-9fd8-4014-b239-9638f3aa9571 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:29.586559 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 514/1013] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:29.595003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db761863-2dc2-44f7-a000-b2f318e6ed9e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:29.595300 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db761863-2dc2-44f7-a000-b2f318e6ed9e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:29.595579 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db761863-2dc2-44f7-a000-b2f318e6ed9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:29.617168 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 500/1014] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:29 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:29.619090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-db761863-2dc2-44f7-a000-b2f318e6ed9e None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:29.620280 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-db761863-2dc2-44f7-a000-b2f318e6ed9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:29.638213 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c183e81-fef0-4d50-b183-39506952fe26 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:29.655248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:29.701573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 515/1015] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:29 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 04:27:29.722402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db761863-2dc2-44f7-a000-b2f318e6ed9e None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:29.757341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c183e81-fef0-4d50-b183-39506952fe26 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:29.803178 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e45541-c731-4999-a23c-0235154355c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:29.804626 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e45541-c731-4999-a23c-0235154355c7 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:29.805116 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e45541-c731-4999-a23c-0235154355c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:29.806184 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e45541-c731-4999-a23c-0235154355c7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:29.807816 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e45541-c731-4999-a23c-0235154355c7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:29.809645 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e45541-c731-4999-a23c-0235154355c7 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:29.822193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-68e45541-c731-4999-a23c-0235154355c7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 04:27:29.822193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:29.822193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:27:29.822193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:27:29.822193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:27:29.822193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:27:29.822193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:27:29.822193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:29.822193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 04:27:29.822193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:29.822193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:27:29.822193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:29.822193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:29.822193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:29.822193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:29.822193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:29.822193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:29.823458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:29.823458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:29.823458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:29.823458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:29.823458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:29.823458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:29.823458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:27:29.823458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:27:29.823458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:27:29.823458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:27:29.823458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:29.823458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:29.823458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:29.823458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:29.823458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:29.823458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:29.824490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:29.824490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:29.824490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:29.824490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:29.824490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:29.824490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:29.824490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:29.824490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:29.824490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:29.824490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:29.824490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:27:29.824490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:29.824490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 04:27:29.824490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:29.828411 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 501/1016] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:27:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:29.835315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e45541-c731-4999-a23c-0235154355c7 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:29.869606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db761863-2dc2-44f7-a000-b2f318e6ed9e None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:29.871286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:29.908400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb0963-54b3-460f-a88b-bbd3d23b1de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:29.909074 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb0963-54b3-460f-a88b-bbd3d23b1de6 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:29.909293 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb0963-54b3-460f-a88b-bbd3d23b1de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:29.909594 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb0963-54b3-460f-a88b-bbd3d23b1de6 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:29.910301 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb0963-54b3-460f-a88b-bbd3d23b1de6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:29.911650 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb0963-54b3-460f-a88b-bbd3d23b1de6 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:29.930228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 516/1017] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Aug 4 04:27:29 2022] POST /identity/v3/services => generated 242 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 04:27:29.936140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:29.964875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e45541-c731-4999-a23c-0235154355c7 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:30.011841 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c9ec8-d7fd-4c70-9070-d531615b21a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:30.012524 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c9ec8-d7fd-4c70-9070-d531615b21a2 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:30.012804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c9ec8-d7fd-4c70-9070-d531615b21a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:30.013072 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c9ec8-d7fd-4c70-9070-d531615b21a2 None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:30.013791 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c9ec8-d7fd-4c70-9070-d531615b21a2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:30.014117 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c9ec8-d7fd-4c70-9070-d531615b21a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:30.016132 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c9ec8-d7fd-4c70-9070-d531615b21a2 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:30.026353 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 502/1018] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Aug 4 04:27:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:30.027508 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb0963-54b3-460f-a88b-bbd3d23b1de6 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:30.139546 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb0963-54b3-460f-a88b-bbd3d23b1de6 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:30.175351 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef4b65-f0f6-4f5e-8e2d-1bc854da22ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:30.175977 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef4b65-f0f6-4f5e-8e2d-1bc854da22ff None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:30.176260 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef4b65-f0f6-4f5e-8e2d-1bc854da22ff None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:30.176533 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef4b65-f0f6-4f5e-8e2d-1bc854da22ff None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:30.177212 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef4b65-f0f6-4f5e-8e2d-1bc854da22ff None admin] RBAC: Adding query filter params (type=compute) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:30.177517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef4b65-f0f6-4f5e-8e2d-1bc854da22ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:30.178910 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef4b65-f0f6-4f5e-8e2d-1bc854da22ff None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:30.184182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 517/1019] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Aug 4 04:27:30 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:30.190375 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c9ec8-d7fd-4c70-9070-d531615b21a2 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:30.292265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c9ec8-d7fd-4c70-9070-d531615b21a2 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:30.326562 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7fedd-fa34-415d-8172-1d174cec7947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:30.327722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7fedd-fa34-415d-8172-1d174cec7947 None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:30.328190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7fedd-fa34-415d-8172-1d174cec7947 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:30.328604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7fedd-fa34-415d-8172-1d174cec7947 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:30.329324 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7fedd-fa34-415d-8172-1d174cec7947 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:30.330706 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7fedd-fa34-415d-8172-1d174cec7947 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:30.336047 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:30.352642 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 503/1020] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:27:30 2022] POST /identity/v3/endpoints => generated 334 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:27:31.472089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ad1ce-8818-4441-a7cb-dab6e9f1f3e5 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:31.472612 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ad1ce-8818-4441-a7cb-dab6e9f1f3e5 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:31.472689 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ad1ce-8818-4441-a7cb-dab6e9f1f3e5 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:31.474199 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 518/1021] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:31.479368 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb93d2-ea8b-40a1-948a-d8d31c51d121 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:31.479782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb93d2-ea8b-40a1-948a-d8d31c51d121 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:31.479953 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb93d2-ea8b-40a1-948a-d8d31c51d121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:31.501865 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-e3eb93d2-ea8b-40a1-948a-d8d31c51d121 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:31.501865 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-e3eb93d2-ea8b-40a1-948a-d8d31c51d121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:31.581964 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 504/1022] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:31 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:31.607689 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ad1ce-8818-4441-a7cb-dab6e9f1f3e5 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:31.637039 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:31.715677 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ad1ce-8818-4441-a7cb-dab6e9f1f3e5 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:31.749474 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309bb4b9-71f9-44c7-b02b-0fb3fb4acd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:31.750045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bb4b9-71f9-44c7-b02b-0fb3fb4acd5c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:31.750333 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bb4b9-71f9-44c7-b02b-0fb3fb4acd5c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:31.750597 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bb4b9-71f9-44c7-b02b-0fb3fb4acd5c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:31.751775 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bb4b9-71f9-44c7-b02b-0fb3fb4acd5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:31.753209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bb4b9-71f9-44c7-b02b-0fb3fb4acd5c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:31.755032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 519/1023] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:31.760498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb93d2-ea8b-40a1-948a-d8d31c51d121 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:31.860843 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb93d2-ea8b-40a1-948a-d8d31c51d121 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:31.878880 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:31.896861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05aeeee-0af9-4355-93a7-e5854c796d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:31.897466 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05aeeee-0af9-4355-93a7-e5854c796d1a None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:31.897784 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05aeeee-0af9-4355-93a7-e5854c796d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:31.898055 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05aeeee-0af9-4355-93a7-e5854c796d1a None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:31.898750 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05aeeee-0af9-4355-93a7-e5854c796d1a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:31.900159 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05aeeee-0af9-4355-93a7-e5854c796d1a None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:31.923015 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 505/1024] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Aug 4 04:27:31 2022] POST /identity/v3/users => generated 285 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:27:32.026676 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d948c9-afaf-4f3b-abbb-fef2feca2420 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:32.027002 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d948c9-afaf-4f3b-abbb-fef2feca2420 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:32.027171 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d948c9-afaf-4f3b-abbb-fef2feca2420 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:32.028692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 520/1025] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:32.034899 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017178d-7729-4717-ab02-dce9e99b4cf4 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:32.035203 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017178d-7729-4717-ab02-dce9e99b4cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:32.035454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017178d-7729-4717-ab02-dce9e99b4cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:32.055359 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-c017178d-7729-4717-ab02-dce9e99b4cf4 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:32.056560 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c017178d-7729-4717-ab02-dce9e99b4cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:32.132067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 506/1026] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:32.154288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d948c9-afaf-4f3b-abbb-fef2feca2420 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:32.256753 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:32.265388 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d948c9-afaf-4f3b-abbb-fef2feca2420 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:32.299190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b74d0-28c5-4363-ad1a-5378c3fa1c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:32.299813 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b74d0-28c5-4363-ad1a-5378c3fa1c03 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:32.300087 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b74d0-28c5-4363-ad1a-5378c3fa1c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:32.300353 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b74d0-28c5-4363-ad1a-5378c3fa1c03 None admin] PATH_INFO: `/v3/services/image` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:32.300979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9b74d0-28c5-4363-ad1a-5378c3fa1c03 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:32.302417 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9b74d0-28c5-4363-ad1a-5378c3fa1c03 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:32.311368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-9f9b74d0-28c5-4363-ad1a-5378c3fa1c03 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 04:27:32.311368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:32.311368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:27:32.311368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:27:32.311368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:27:32.311368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:27:32.311368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:27:32.311368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:32.311368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 04:27:32.311368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:32.311368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:27:32.311368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:32.311368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:32.311368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:32.311368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:32.311368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:32.311368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:32.312963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:32.312963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:32.312963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:32.312963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:32.312963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:32.312963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:32.312963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:27:32.312963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:27:32.312963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:27:32.312963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:27:32.312963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:32.312963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:32.312963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:32.312963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:32.312963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:32.312963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:32.313861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:32.313861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:32.313861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:32.313861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:32.313861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:32.313861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:32.313861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:32.313861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:32.313861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:32.313861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:32.313861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:27:32.313861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:32.313861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 04:27:32.313861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:32.313861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 521/1027] 10.222.0.47 () {56 vars in 1105 bytes} [Thu Aug 4 04:27:32 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:32.320995 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c017178d-7729-4717-ab02-dce9e99b4cf4 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:32.424588 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c017178d-7729-4717-ab02-dce9e99b4cf4 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:32.426932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:32.462327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1340a-45d0-44eb-9d27-e219c02d7479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:32.464400 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1340a-45d0-44eb-9d27-e219c02d7479 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:32.464400 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1340a-45d0-44eb-9d27-e219c02d7479 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:32.464400 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1340a-45d0-44eb-9d27-e219c02d7479 None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:32.464400 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1340a-45d0-44eb-9d27-e219c02d7479 None admin] RBAC: Adding query filter params (name=image) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:32.464717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1340a-45d0-44eb-9d27-e219c02d7479 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:32.465809 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1340a-45d0-44eb-9d27-e219c02d7479 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:32.471732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 507/1028] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:32.477286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b74d0-28c5-4363-ad1a-5378c3fa1c03 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:32.479480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bd3a5-1b99-4884-a93c-08bdc98abf6f None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:32.479862 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bd3a5-1b99-4884-a93c-08bdc98abf6f None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:32.480034 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bd3a5-1b99-4884-a93c-08bdc98abf6f None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:32.481460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 508/1029] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:27:32.486768 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05e521-8d9f-4a2f-a122-95cbccc9908b None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:32.486768 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05e521-8d9f-4a2f-a122-95cbccc9908b None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:32.486768 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05e521-8d9f-4a2f-a122-95cbccc9908b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:32.504595 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-1e05e521-8d9f-4a2f-a122-95cbccc9908b None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:32.506061 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-1e05e521-8d9f-4a2f-a122-95cbccc9908b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:32.581675 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 509/1030] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 04:27:32.601232 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b74d0-28c5-4363-ad1a-5378c3fa1c03 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:32.604404 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05e521-8d9f-4a2f-a122-95cbccc9908b None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:32.637790 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac01601-1f8e-43d9-9860-fa1ccc493515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:32.638407 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac01601-1f8e-43d9-9860-fa1ccc493515 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:32.638685 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac01601-1f8e-43d9-9860-fa1ccc493515 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:32.638956 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac01601-1f8e-43d9-9860-fa1ccc493515 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:32.640552 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac01601-1f8e-43d9-9860-fa1ccc493515 None admin] RBAC: Adding query filter params (type=image) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:32.640969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac01601-1f8e-43d9-9860-fa1ccc493515 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:32.642360 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac01601-1f8e-43d9-9860-fa1ccc493515 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:32.648228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 522/1031] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:32 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:32.653930 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac01601-1f8e-43d9-9860-fa1ccc493515 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:32.680280 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:32.710397 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:32.714690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05e521-8d9f-4a2f-a122-95cbccc9908b None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:32.753709 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b45957-9548-4355-8811-39bc587cfc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:32.754353 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b45957-9548-4355-8811-39bc587cfc08 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:32.754632 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b45957-9548-4355-8811-39bc587cfc08 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:32.754896 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b45957-9548-4355-8811-39bc587cfc08 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:32.762669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac01601-1f8e-43d9-9860-fa1ccc493515 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:32.766510 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b45957-9548-4355-8811-39bc587cfc08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:27:32.771517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-e0b45957-9548-4355-8811-39bc587cfc08 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 04:27:32.771517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:32.771517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:32.771517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:32.771517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:32.771517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:32.771517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:32.771517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:32.771517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:32.771517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:32.771517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:32.771517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:32.771517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:27:32.771517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:27:32.771517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:27:32.771517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:27:32.772410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:27:32.772410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:27:32.772410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:27:32.772410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:27:32.772410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:32.772410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:32.772410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:32.772410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:32.772410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:32.772410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:32.772410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:32.772410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:32.772410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:32.772410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:32.772410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:32.772410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:32.773248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:32.773248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:32.773248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:32.773248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:32.773248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:27:32.773248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:27:32.773248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:27:32.773248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:27:32.773248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:27:32.773248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:27:32.773248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 04:27:32.773248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:32.773248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 510/1032] 10.222.0.47 () {56 vars in 1120 bytes} [Thu Aug 4 04:27:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:32.778605 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b45957-9548-4355-8811-39bc587cfc08 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:32.807925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0309be0-63a4-4027-bb85-e5d198b0fea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:32.808549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0309be0-63a4-4027-bb85-e5d198b0fea9 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:32.808837 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0309be0-63a4-4027-bb85-e5d198b0fea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:32.809109 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0309be0-63a4-4027-bb85-e5d198b0fea9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:32.809897 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0309be0-63a4-4027-bb85-e5d198b0fea9 None admin] RBAC: Adding query filter params (interface=public, service_id=ec317fa706094775848f430d3e8f873f, region_id=RegionOne) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:32.810206 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0309be0-63a4-4027-bb85-e5d198b0fea9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:32.811746 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0309be0-63a4-4027-bb85-e5d198b0fea9 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:32.817082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 523/1033] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Aug 4 04:27:32 2022] GET /identity/v3/endpoints?service_id=ec317fa706094775848f430d3e8f873f&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:32.822414 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0309be0-63a4-4027-bb85-e5d198b0fea9 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:32.889646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b45957-9548-4355-8811-39bc587cfc08 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:32.909249 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:32.926125 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e97ee-a3e7-485c-b1fa-204fd373ce13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:32.926746 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e97ee-a3e7-485c-b1fa-204fd373ce13 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:32.927022 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e97ee-a3e7-485c-b1fa-204fd373ce13 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:32.927292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e97ee-a3e7-485c-b1fa-204fd373ce13 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:32.927660 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0309be0-63a4-4027-bb85-e5d198b0fea9 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:32.927999 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e97ee-a3e7-485c-b1fa-204fd373ce13 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:32.928299 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e97ee-a3e7-485c-b1fa-204fd373ce13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:32.929677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e97ee-a3e7-485c-b1fa-204fd373ce13 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:32.942815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 511/1034] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Aug 4 04:27:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:32.943823 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e97ee-a3e7-485c-b1fa-204fd373ce13 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:32.964735 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2310e2-c342-4ae3-9a38-58130aeeaad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:32.965354 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2310e2-c342-4ae3-9a38-58130aeeaad1 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:32.965670 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2310e2-c342-4ae3-9a38-58130aeeaad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:32.965947 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2310e2-c342-4ae3-9a38-58130aeeaad1 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:32.966814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2310e2-c342-4ae3-9a38-58130aeeaad1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:32.967274 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:32.968767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2310e2-c342-4ae3-9a38-58130aeeaad1 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:32.976556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 524/1035] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Aug 4 04:27:32 2022] GET /identity/v3/services => generated 1957 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 04:27:33.057171 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e97ee-a3e7-485c-b1fa-204fd373ce13 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:33.094114 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a1fce-7dd8-464d-80e5-849aeef26a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:33.094771 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a1fce-7dd8-464d-80e5-849aeef26a4c None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:33.095048 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a1fce-7dd8-464d-80e5-849aeef26a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:33.095316 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a1fce-7dd8-464d-80e5-849aeef26a4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:33.107162 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a1fce-7dd8-464d-80e5-849aeef26a4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:33.109181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a1fce-7dd8-464d-80e5-849aeef26a4c None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:33.124738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-315a1fce-7dd8-464d-80e5-849aeef26a4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:33.124738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:33.124738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:33.124738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:33.124738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:33.124738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:33.124738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:33.124738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:33.124738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:33.124738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:33.124738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:33.124738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:33.124738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:27:33.124738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:27:33.124738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:27:33.125904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:27:33.125904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:33.125904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:33.125904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:33.125904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:33.125904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:33.125904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:33.125904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:33.125904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:33.125904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:33.125904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:33.125904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:33.125904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:33.125904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:33.125904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:33.125904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:33.125904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:33.126808 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:27:33.126808 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:27:33.126808 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:33.126808 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:33.126808 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 512/1036] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:33.132919 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2310e2-c342-4ae3-9a38-58130aeeaad1 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:33.246151 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2310e2-c342-4ae3-9a38-58130aeeaad1 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:33.280948 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430356f-58b9-4112-90e9-b576475622ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:33.281668 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430356f-58b9-4112-90e9-b576475622ac None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:33.281949 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430356f-58b9-4112-90e9-b576475622ac None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:33.282219 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430356f-58b9-4112-90e9-b576475622ac None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:33.282907 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430356f-58b9-4112-90e9-b576475622ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:33.283211 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430356f-58b9-4112-90e9-b576475622ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:33.284568 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430356f-58b9-4112-90e9-b576475622ac None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:33.296577 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 525/1037] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:33.302430 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a1fce-7dd8-464d-80e5-849aeef26a4c None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:33.372286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:33.403741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a1fce-7dd8-464d-80e5-849aeef26a4c None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:33.439713 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2624ae2-3d1c-4040-af9b-a10702595874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:33.440351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2624ae2-3d1c-4040-af9b-a10702595874 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:33.440665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2624ae2-3d1c-4040-af9b-a10702595874 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:33.440960 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2624ae2-3d1c-4040-af9b-a10702595874 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:33.449063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2624ae2-3d1c-4040-af9b-a10702595874 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:33.450517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2624ae2-3d1c-4040-af9b-a10702595874 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:33.461874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-b2624ae2-3d1c-4040-af9b-a10702595874 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 04:27:33.461874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:33.461874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:33.461874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:33.461874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:33.461874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:33.461874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:33.461874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:33.461874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:33.461874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:33.461874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:33.461874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:33.461874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:27:33.461874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:27:33.461874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:27:33.461874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:27:33.462889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:33.462889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:33.462889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:27:33.462889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:33.462889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:27:33.462889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:33.462889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:33.462889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:33.462889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:33.462889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:33.462889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:33.462889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:33.462889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:33.462889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:33.462889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:33.462889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:33.462889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:33.462889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:33.463816 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:33.463816 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:33.463816 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:27:33.463816 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:27:33.463816 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:27:33.463816 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:27:33.463816 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:27:33.463816 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:27:33.463816 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 04:27:33.463816 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:33.465933 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 513/1038] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Aug 4 04:27:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:33.469053 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430356f-58b9-4112-90e9-b576475622ac None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:33.481896 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:33.584341 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430356f-58b9-4112-90e9-b576475622ac None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:33.619148 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3255a-09f2-41d6-9a4f-48bde3d3f30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:33.619766 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3255a-09f2-41d6-9a4f-48bde3d3f30e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:33.620043 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3255a-09f2-41d6-9a4f-48bde3d3f30e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:33.620307 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3255a-09f2-41d6-9a4f-48bde3d3f30e None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:33.621095 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3255a-09f2-41d6-9a4f-48bde3d3f30e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:33.621395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3255a-09f2-41d6-9a4f-48bde3d3f30e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:33.622841 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3255a-09f2-41d6-9a4f-48bde3d3f30e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:33.641432 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 526/1039] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Aug 4 04:27:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:33.646877 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2624ae2-3d1c-4040-af9b-a10702595874 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:33.748749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2624ae2-3d1c-4040-af9b-a10702595874 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:33.778013 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:33.784744 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0faef03-d14e-4309-ba0b-dcbf90714bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:33.785429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0faef03-d14e-4309-ba0b-dcbf90714bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:33.785667 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0faef03-d14e-4309-ba0b-dcbf90714bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:33.785935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0faef03-d14e-4309-ba0b-dcbf90714bc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:33.852920 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0faef03-d14e-4309-ba0b-dcbf90714bc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:33.854375 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0faef03-d14e-4309-ba0b-dcbf90714bc1 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:33.957035 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 514/1040] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:27:33 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 04:27:33.958516 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3255a-09f2-41d6-9a4f-48bde3d3f30e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:34.025281 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:34.060556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3255a-09f2-41d6-9a4f-48bde3d3f30e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:34.083197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2d623-6197-48e7-9bd2-22dbd0c73930 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:34.083540 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2d623-6197-48e7-9bd2-22dbd0c73930 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:34.083764 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2d623-6197-48e7-9bd2-22dbd0c73930 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:34.085259 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 515/1041] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:27:34.090174 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb9aba-5bba-428b-bf78-9bd164676bf7 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:34.090463 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb9aba-5bba-428b-bf78-9bd164676bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:34.090740 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb9aba-5bba-428b-bf78-9bd164676bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:34.098448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985153b-c5e0-4495-bcd4-f41e690096af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:34.100043 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985153b-c5e0-4495-bcd4-f41e690096af None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:34.100320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985153b-c5e0-4495-bcd4-f41e690096af None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:34.100591 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985153b-c5e0-4495-bcd4-f41e690096af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:34.110392 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-13eb9aba-5bba-428b-bf78-9bd164676bf7 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:34.111734 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-13eb9aba-5bba-428b-bf78-9bd164676bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:34.112753 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3985153b-c5e0-4495-bcd4-f41e690096af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:34.114302 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3985153b-c5e0-4495-bcd4-f41e690096af None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:34.129908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-3985153b-c5e0-4495-bcd4-f41e690096af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:34.129908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:34.129908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:34.129908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:34.129908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:34.129908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:34.129908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:34.129908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:34.129908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:34.129908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:34.129908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:34.129908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:34.129908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:27:34.129908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:27:34.129908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:27:34.131001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:27:34.131001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:34.131001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:34.131001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:34.131001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:34.131001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:34.131001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:34.131001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:34.131001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:34.131001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:34.131001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:34.131001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:34.131001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:34.131001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:34.131001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:34.131001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:34.131001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:34.132066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:27:34.132066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:27:34.132066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:34.132066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:34.135876 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 527/1042] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:34.136718 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985153b-c5e0-4495-bcd4-f41e690096af None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:34.143198 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:34.189941 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 516/1043] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 04:27:34.209409 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb9aba-5bba-428b-bf78-9bd164676bf7 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:34.246723 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985153b-c5e0-4495-bcd4-f41e690096af None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:34.268753 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:34.287831 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054030a8-6089-4fa6-b92d-8a48a7b1d196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:34.288436 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054030a8-6089-4fa6-b92d-8a48a7b1d196 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:34.288712 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054030a8-6089-4fa6-b92d-8a48a7b1d196 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:34.288978 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054030a8-6089-4fa6-b92d-8a48a7b1d196 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:34.289700 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054030a8-6089-4fa6-b92d-8a48a7b1d196 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:34.290002 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054030a8-6089-4fa6-b92d-8a48a7b1d196 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:34.291355 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054030a8-6089-4fa6-b92d-8a48a7b1d196 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:34.303204 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 528/1044] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:34.309093 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054030a8-6089-4fa6-b92d-8a48a7b1d196 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:34.315867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb9aba-5bba-428b-bf78-9bd164676bf7 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:34.350035 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf0ea5-140a-432c-8d89-db8c578cff4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:34.350810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf0ea5-140a-432c-8d89-db8c578cff4c None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:34.351215 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf0ea5-140a-432c-8d89-db8c578cff4c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:34.351594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf0ea5-140a-432c-8d89-db8c578cff4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:34.363129 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf0ea5-140a-432c-8d89-db8c578cff4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:27:34.368227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-abaf0ea5-140a-432c-8d89-db8c578cff4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:27:34.368227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:34.368227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:34.368227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:34.368227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:34.368227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:34.368227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:34.368227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:34.368227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:34.368227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:34.368227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:34.368227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:34.368227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:27:34.368227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:27:34.368227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:27:34.368227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:27:34.369351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:27:34.369351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:27:34.369351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:27:34.369351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:27:34.369351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:34.369351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:34.369351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:34.369351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:34.369351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:34.369351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:34.369351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:34.369351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:34.369351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:34.369351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:34.369351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:34.369351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:34.370239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:34.370239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:34.370239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:34.370239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:34.370239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:27:34.370239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:27:34.370239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:27:34.370239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:27:34.370239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:27:34.370239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:27:34.370239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:27:34.370239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:34.370239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 517/1045] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:27:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:34.378686 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf0ea5-140a-432c-8d89-db8c578cff4c None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:34.403673 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:34.423064 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054030a8-6089-4fa6-b92d-8a48a7b1d196 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:34.460453 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4cfac-6f1d-4392-a806-9cdecc23c322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:34.461098 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4cfac-6f1d-4392-a806-9cdecc23c322 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:34.461814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4cfac-6f1d-4392-a806-9cdecc23c322 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:34.462101 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4cfac-6f1d-4392-a806-9cdecc23c322 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:34.470316 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4cfac-6f1d-4392-a806-9cdecc23c322 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:34.471832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4cfac-6f1d-4392-a806-9cdecc23c322 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:34.486873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-8ae4cfac-6f1d-4392-a806-9cdecc23c322 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:34.486873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:34.486873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:34.486873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:34.486873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:34.486873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:34.486873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:34.486873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:34.486873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:34.486873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:34.486873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:34.486873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:34.486873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:27:34.486873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:27:34.486873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:27:34.493164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:27:34.493164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:34.493164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:34.493164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:34.493164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:34.493164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:34.493164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:34.493164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:34.493164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:34.493164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:34.493164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:34.493164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:34.493164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:34.493164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:34.493164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:34.493164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:34.493164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:34.494106 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:27:34.494106 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:27:34.494106 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:27:34.494106 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:27:34.494106 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:27:34.494106 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:27:34.494106 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:34.494106 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:34.494544 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf0ea5-140a-432c-8d89-db8c578cff4c None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:34.494618 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 529/1046] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:27:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:34.494618 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4cfac-6f1d-4392-a806-9cdecc23c322 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:34.527018 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94daf58-6c42-416f-acb8-5a8eca6a6e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:34.527894 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94daf58-6c42-416f-acb8-5a8eca6a6e71 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:34.528178 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94daf58-6c42-416f-acb8-5a8eca6a6e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:34.528455 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94daf58-6c42-416f-acb8-5a8eca6a6e71 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:34.529178 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94daf58-6c42-416f-acb8-5a8eca6a6e71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:34.529483 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94daf58-6c42-416f-acb8-5a8eca6a6e71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:34.531285 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94daf58-6c42-416f-acb8-5a8eca6a6e71 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:34.540339 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 518/1047] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:27:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:34.546456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94daf58-6c42-416f-acb8-5a8eca6a6e71 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:34.552947 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:34.618114 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4cfac-6f1d-4392-a806-9cdecc23c322 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:34.641305 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:34.656671 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edafd6ef-e95c-41c6-a9e9-a26f21c0beaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:34.657293 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edafd6ef-e95c-41c6-a9e9-a26f21c0beaa None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:34.657596 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edafd6ef-e95c-41c6-a9e9-a26f21c0beaa None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:34.657872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edafd6ef-e95c-41c6-a9e9-a26f21c0beaa None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:34.658623 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edafd6ef-e95c-41c6-a9e9-a26f21c0beaa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:34.658932 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edafd6ef-e95c-41c6-a9e9-a26f21c0beaa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:34.659028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94daf58-6c42-416f-acb8-5a8eca6a6e71 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:34.660476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edafd6ef-e95c-41c6-a9e9-a26f21c0beaa None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:34.674579 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 530/1048] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:27:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:34.680294 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edafd6ef-e95c-41c6-a9e9-a26f21c0beaa None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:34.696871 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ee2d1-1ef8-4a01-8690-1ac9d4c39633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:34.697576 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ee2d1-1ef8-4a01-8690-1ac9d4c39633 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:34.697848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ee2d1-1ef8-4a01-8690-1ac9d4c39633 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:34.698122 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ee2d1-1ef8-4a01-8690-1ac9d4c39633 None admin] PATH_INFO: `/v3/users/455d295339864238844175f3302cd8d0` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:34.713856 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ee2d1-1ef8-4a01-8690-1ac9d4c39633 None admin] RBAC: Authorizing `identity:get_user(user_id=455d295339864238844175f3302cd8d0)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:34.715291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ee2d1-1ef8-4a01-8690-1ac9d4c39633 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:34.722337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 519/1049] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:27:34 2022] GET /identity/v3/users/455d295339864238844175f3302cd8d0 => generated 285 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:34.723006 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ee2d1-1ef8-4a01-8690-1ac9d4c39633 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:34.784321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:34.787117 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edafd6ef-e95c-41c6-a9e9-a26f21c0beaa None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:34.827758 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85924e8b-abdf-4b76-a03a-9932abaa2ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:34.828446 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85924e8b-abdf-4b76-a03a-9932abaa2ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:34.828883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85924e8b-abdf-4b76-a03a-9932abaa2ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:34.829159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85924e8b-abdf-4b76-a03a-9932abaa2ec1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:34.830288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85924e8b-abdf-4b76-a03a-9932abaa2ec1 None admin] RBAC: Adding query filter params (role.id=3747dec438cd4349bd9e402694b88574, scope.project.id=f37a5f3140ec431da8ea14568654114a, user.id=caa2901451aa4554bdfcac1bddbc620a) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:34.830606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85924e8b-abdf-4b76-a03a-9932abaa2ec1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:34.832036 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85924e8b-abdf-4b76-a03a-9932abaa2ec1 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:34.832405 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ee2d1-1ef8-4a01-8690-1ac9d4c39633 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:34.838613 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 531/1050] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:27:34 2022] GET /identity/v3/role_assignments?user.id=caa2901451aa4554bdfcac1bddbc620a&scope.project.id=f37a5f3140ec431da8ea14568654114a&role.id=3747dec438cd4349bd9e402694b88574 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:34.872312 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db21dc1c-3bd4-42be-b72c-132e42e15ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:34.872934 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21dc1c-3bd4-42be-b72c-132e42e15ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:34.873215 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21dc1c-3bd4-42be-b72c-132e42e15ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:34.873508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21dc1c-3bd4-42be-b72c-132e42e15ba1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:34.888685 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:34.958480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db21dc1c-3bd4-42be-b72c-132e42e15ba1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:34.959957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db21dc1c-3bd4-42be-b72c-132e42e15ba1 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:35.020407 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8b1c4-0cfc-406b-aace-171a33839f2c None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:35.020714 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8b1c4-0cfc-406b-aace-171a33839f2c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:35.020954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8b1c4-0cfc-406b-aace-171a33839f2c None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:35.022476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 532/1051] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:27:35.027059 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bbee5-5505-4197-8d3b-961400889d55 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:35.027333 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bbee5-5505-4197-8d3b-961400889d55 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:35.027595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bbee5-5505-4197-8d3b-961400889d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:35.046406 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-f90bbee5-5505-4197-8d3b-961400889d55 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:35.047699 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f90bbee5-5505-4197-8d3b-961400889d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:35.075223 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 520/1052] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:27:34 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 04:27:35.076354 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db21dc1c-3bd4-42be-b72c-132e42e15ba1 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:35.120114 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:35.133118 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 533/1053] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 04:27:35.156130 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bbee5-5505-4197-8d3b-961400889d55 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:35.182646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db21dc1c-3bd4-42be-b72c-132e42e15ba1 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:35.222167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f81efd-45d4-45fa-99f0-545d354ce2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:35.222844 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f81efd-45d4-45fa-99f0-545d354ce2fa None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:35.223148 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f81efd-45d4-45fa-99f0-545d354ce2fa None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:35.223407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f81efd-45d4-45fa-99f0-545d354ce2fa None admin] PATH_INFO: `/v3/projects/7f32ef360443455fa75d77dbf975364f` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:35.224580 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f81efd-45d4-45fa-99f0-545d354ce2fa None admin] RBAC: Authorizing `identity:get_project(project_id=7f32ef360443455fa75d77dbf975364f)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:35.226775 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f81efd-45d4-45fa-99f0-545d354ce2fa None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:35.227993 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 521/1054] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:27:35 2022] GET /identity/v3/projects/7f32ef360443455fa75d77dbf975364f => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:35.235195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f81efd-45d4-45fa-99f0-545d354ce2fa None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:35.259185 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:35.266570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bbee5-5505-4197-8d3b-961400889d55 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:35.305281 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71519e6a-8c9b-42bf-9e1a-d71975f8d787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:35.305968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71519e6a-8c9b-42bf-9e1a-d71975f8d787 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:35.306462 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71519e6a-8c9b-42bf-9e1a-d71975f8d787 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:35.306582 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71519e6a-8c9b-42bf-9e1a-d71975f8d787 None admin] PATH_INFO: `/v3/services/image` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:35.307149 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71519e6a-8c9b-42bf-9e1a-d71975f8d787 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:35.308641 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71519e6a-8c9b-42bf-9e1a-d71975f8d787 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:35.317886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-71519e6a-8c9b-42bf-9e1a-d71975f8d787 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 04:27:35.317886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:35.317886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:27:35.317886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:27:35.317886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:27:35.317886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:27:35.317886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:27:35.317886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:35.317886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 04:27:35.317886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:35.317886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:27:35.317886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:35.317886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:35.317886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:35.317886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:35.317886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:35.317886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:35.318890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:35.318890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:35.318890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:35.318890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:35.318890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:35.318890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:35.318890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:27:35.318890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:27:35.318890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:27:35.318890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:27:35.318890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:35.318890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:35.318890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:35.318890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:35.318890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:35.318890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:35.319711 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:35.319711 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:35.319711 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:35.319711 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:35.319711 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:35.319711 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:35.319711 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:35.319711 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:35.319711 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:35.319711 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:35.319711 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:27:35.319711 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:27:35.319711 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 04:27:35.319711 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:35.319711 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 534/1055] 10.222.0.47 () {56 vars in 1105 bytes} [Thu Aug 4 04:27:35 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:35.324772 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71519e6a-8c9b-42bf-9e1a-d71975f8d787 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:35.360506 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f81efd-45d4-45fa-99f0-545d354ce2fa None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:35.373147 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:35.402818 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9929f3e-582a-486b-8328-c332a01b38e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:35.403357 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9929f3e-582a-486b-8328-c332a01b38e9 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:35.403638 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9929f3e-582a-486b-8328-c332a01b38e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:35.403911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9929f3e-582a-486b-8328-c332a01b38e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:35.404701 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9929f3e-582a-486b-8328-c332a01b38e9 None admin] RBAC: Adding query filter params (role.id=4a6517e5890a49718303007dac0756c7, scope.project.id=7f32ef360443455fa75d77dbf975364f, user.id=455d295339864238844175f3302cd8d0) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:35.405127 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9929f3e-582a-486b-8328-c332a01b38e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:35.406568 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9929f3e-582a-486b-8328-c332a01b38e9 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:35.412346 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 522/1056] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:27:35 2022] GET /identity/v3/role_assignments?user.id=455d295339864238844175f3302cd8d0&scope.project.id=7f32ef360443455fa75d77dbf975364f&role.id=4a6517e5890a49718303007dac0756c7 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:35.434534 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71519e6a-8c9b-42bf-9e1a-d71975f8d787 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:35.473720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd291d8-c021-4159-8e86-8543c66db467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:35.474411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd291d8-c021-4159-8e86-8543c66db467 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:35.474779 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd291d8-c021-4159-8e86-8543c66db467 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:35.475050 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd291d8-c021-4159-8e86-8543c66db467 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:35.475790 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd291d8-c021-4159-8e86-8543c66db467 None admin] RBAC: Adding query filter params (name=image) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:35.476098 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd291d8-c021-4159-8e86-8543c66db467 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:35.477472 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd291d8-c021-4159-8e86-8543c66db467 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:35.483832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 535/1057] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:35.489345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9929f3e-582a-486b-8328-c332a01b38e9 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:35.610366 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9929f3e-582a-486b-8328-c332a01b38e9 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:35.644756 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545bede-63be-4a4e-82a9-a03847663aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:35.645370 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545bede-63be-4a4e-82a9-a03847663aaa None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:35.645692 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545bede-63be-4a4e-82a9-a03847663aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:35.645965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545bede-63be-4a4e-82a9-a03847663aaa None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:35.646637 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545bede-63be-4a4e-82a9-a03847663aaa None admin] RBAC: Adding query filter params (type=image) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:35.646936 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545bede-63be-4a4e-82a9-a03847663aaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:35.648343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545bede-63be-4a4e-82a9-a03847663aaa None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:35.657684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 523/1058] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:35 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:35.660160 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd291d8-c021-4159-8e86-8543c66db467 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:35.686607 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:35.768003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd291d8-c021-4159-8e86-8543c66db467 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:35.802142 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c67f0-2366-4cdb-9d53-e705fdc336ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:35.802767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c67f0-2366-4cdb-9d53-e705fdc336ff None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:35.803051 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c67f0-2366-4cdb-9d53-e705fdc336ff None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:35.803328 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c67f0-2366-4cdb-9d53-e705fdc336ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:35.804045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c67f0-2366-4cdb-9d53-e705fdc336ff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:35.805413 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c67f0-2366-4cdb-9d53-e705fdc336ff None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:35.829904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 536/1059] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:27:35 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:27:36.873219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8888a4-afc0-4940-96e5-a29eb692553f None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:36.874308 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8888a4-afc0-4940-96e5-a29eb692553f None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:36.874676 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8888a4-afc0-4940-96e5-a29eb692553f None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:36.876300 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 524/1060] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:36.881424 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299f2ab-8ae1-4270-96c9-7c7a1309f2c7 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:36.881851 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299f2ab-8ae1-4270-96c9-7c7a1309f2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:36.882026 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299f2ab-8ae1-4270-96c9-7c7a1309f2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:36.900740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-f299f2ab-8ae1-4270-96c9-7c7a1309f2c7 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:36.901967 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f299f2ab-8ae1-4270-96c9-7c7a1309f2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:36.979402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 537/1061] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:37.002002 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8888a4-afc0-4940-96e5-a29eb692553f None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:37.044657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:37.109539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8888a4-afc0-4940-96e5-a29eb692553f None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:37.143892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1540a1-e71f-4614-96f7-ff78070e11c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:37.144456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1540a1-e71f-4614-96f7-ff78070e11c0 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:37.144735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1540a1-e71f-4614-96f7-ff78070e11c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:37.145008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1540a1-e71f-4614-96f7-ff78070e11c0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:37.156452 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1540a1-e71f-4614-96f7-ff78070e11c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:27:37.161330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-ba1540a1-e71f-4614-96f7-ff78070e11c0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 04:27:37.161330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:37.161330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:37.161330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:37.161330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:37.161330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:37.161330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:37.161330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:37.161330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:37.161330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:37.161330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:37.161330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:37.161330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:27:37.161330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:27:37.161330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:27:37.161330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:27:37.162380 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:27:37.162380 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:27:37.162380 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:27:37.162380 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:27:37.162380 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:37.162380 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:37.162380 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:37.162380 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:37.162380 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:37.162380 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:37.162380 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:37.162380 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:37.162380 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:37.162380 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:37.162380 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:37.162380 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:37.163235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:37.163235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:37.163235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:37.163235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:37.163235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:27:37.163235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:27:37.163235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:27:37.163235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:27:37.163235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:27:37.163235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:27:37.163235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 04:27:37.163235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:37.165634 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 525/1062] 10.222.0.47 () {56 vars in 1120 bytes} [Thu Aug 4 04:27:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:37.168787 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299f2ab-8ae1-4270-96c9-7c7a1309f2c7 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:37.173197 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:37.276282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299f2ab-8ae1-4270-96c9-7c7a1309f2c7 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:37.312237 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a09a9-1c2a-4c65-a8a3-c165346e3ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:37.313119 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a09a9-1c2a-4c65-a8a3-c165346e3ebb None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:37.313401 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a09a9-1c2a-4c65-a8a3-c165346e3ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:37.313707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a09a9-1c2a-4c65-a8a3-c165346e3ebb None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:37.314440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a09a9-1c2a-4c65-a8a3-c165346e3ebb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:37.314748 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a09a9-1c2a-4c65-a8a3-c165346e3ebb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:37.316121 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a09a9-1c2a-4c65-a8a3-c165346e3ebb None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:37.329707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 538/1063] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Aug 4 04:27:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:37.332183 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1540a1-e71f-4614-96f7-ff78070e11c0 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:37.433486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:37.438619 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1540a1-e71f-4614-96f7-ff78070e11c0 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:37.461798 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519cf55-6301-4c8c-a3b8-cfa74c299097 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:37.462283 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519cf55-6301-4c8c-a3b8-cfa74c299097 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:37.462565 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519cf55-6301-4c8c-a3b8-cfa74c299097 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:37.464151 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 539/1064] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:27:37.470483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8a328-9a33-444f-badc-297f011aa325 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:37.470766 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8a328-9a33-444f-badc-297f011aa325 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:37.471039 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8a328-9a33-444f-badc-297f011aa325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:37.475275 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45bf8c-5b60-4039-b892-69c3b17c06f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:37.475932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45bf8c-5b60-4039-b892-69c3b17c06f6 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:37.476244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45bf8c-5b60-4039-b892-69c3b17c06f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:37.476550 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45bf8c-5b60-4039-b892-69c3b17c06f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:37.488616 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af45bf8c-5b60-4039-b892-69c3b17c06f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:37.489023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-8cd8a328-9a33-444f-badc-297f011aa325 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:37.490223 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af45bf8c-5b60-4039-b892-69c3b17c06f6 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:37.490327 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8cd8a328-9a33-444f-badc-297f011aa325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:37.505847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-af45bf8c-5b60-4039-b892-69c3b17c06f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:37.505847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:37.505847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:37.505847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:37.505847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:37.505847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:37.505847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:37.505847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:37.505847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:37.505847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:37.505847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:37.505847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:37.505847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:27:37.505847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:27:37.505847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:27:37.506729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:27:37.506729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:37.506729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:37.506729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:37.506729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:37.506729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:37.506729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:37.506729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:37.506729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:37.506729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:37.506729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:37.506729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:37.506729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:37.506729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:37.506729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:37.506729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:37.506729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:37.507623 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:27:37.507623 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:27:37.507623 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:37.507623 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:37.507623 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 526/1065] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:37.512821 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45bf8c-5b60-4039-b892-69c3b17c06f6 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:37.575143 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:37.589992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 540/1066] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 04:27:37.615364 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8a328-9a33-444f-badc-297f011aa325 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:37.627601 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45bf8c-5b60-4039-b892-69c3b17c06f6 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:37.664403 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff69f1-ecef-409d-9421-40248d755023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:37.665232 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff69f1-ecef-409d-9421-40248d755023 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:37.665519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff69f1-ecef-409d-9421-40248d755023 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:37.665823 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff69f1-ecef-409d-9421-40248d755023 None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:37.666652 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff69f1-ecef-409d-9421-40248d755023 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:37.666960 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff69f1-ecef-409d-9421-40248d755023 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:37.668326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff69f1-ecef-409d-9421-40248d755023 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:37.688745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 527/1067] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:27:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:37.690395 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff69f1-ecef-409d-9421-40248d755023 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:37.703130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:37.730848 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8a328-9a33-444f-badc-297f011aa325 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:37.772479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaf7f7-bc62-4e53-be9f-8ae20932b4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:37.773380 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaf7f7-bc62-4e53-be9f-8ae20932b4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:37.773703 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaf7f7-bc62-4e53-be9f-8ae20932b4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:37.773986 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaf7f7-bc62-4e53-be9f-8ae20932b4d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:37.784892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadaf7f7-bc62-4e53-be9f-8ae20932b4d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:27:37.789887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-cadaf7f7-bc62-4e53-be9f-8ae20932b4d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:27:37.789887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:37.789887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:37.789887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:37.789887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:37.789887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:37.789887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:37.789887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:37.789887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:37.789887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:37.789887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:37.789887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:37.789887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:27:37.789887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:27:37.789887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:27:37.789887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:27:37.790814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:27:37.790814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:27:37.790814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:27:37.790814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:27:37.790814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:37.790814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:37.790814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:37.790814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:37.790814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:37.790814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:37.790814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:37.790814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:37.790814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:37.790814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:37.790814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:37.790814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:37.794740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:37.794740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:37.794740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:37.794740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:37.794740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:27:37.794740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:27:37.794740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:27:37.794740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:27:37.794740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:27:37.794740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:27:37.794740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:27:37.794740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:37.794740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 541/1068] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:27:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:37.797814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaf7f7-bc62-4e53-be9f-8ae20932b4d1 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:37.801232 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff69f1-ecef-409d-9421-40248d755023 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:37.840548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735a619-7420-4b15-869e-10625cb2f2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:37.841198 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735a619-7420-4b15-869e-10625cb2f2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:37.841477 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735a619-7420-4b15-869e-10625cb2f2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:37.841782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735a619-7420-4b15-869e-10625cb2f2e5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:37.850163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735a619-7420-4b15-869e-10625cb2f2e5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:37.851623 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735a619-7420-4b15-869e-10625cb2f2e5 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:37.858824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:37.865410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-9735a619-7420-4b15-869e-10625cb2f2e5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 04:27:37.865410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:37.865410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:37.865410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:37.865410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:37.865410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:37.865410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:37.865410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:37.865410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:37.865410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:37.865410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:37.865410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:37.865410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:27:37.865410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:27:37.865410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:27:37.865410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:27:37.866410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:37.866410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:37.866410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:27:37.866410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:37.866410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:27:37.866410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:37.866410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:37.866410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:37.866410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:37.866410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:37.866410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:37.866410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:37.866410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:37.866410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:37.866410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:37.866410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:37.866410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:37.866410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:37.867369 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:37.867369 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:37.867369 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:27:37.867369 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:27:37.867369 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:27:37.867369 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:27:37.867369 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:27:37.867369 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:27:37.867369 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 04:27:37.867369 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:37.867369 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 528/1069] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Aug 4 04:27:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:37.872105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735a619-7420-4b15-869e-10625cb2f2e5 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:37.913853 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaf7f7-bc62-4e53-be9f-8ae20932b4d1 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:37.942526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:37.947970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2318bb-0930-470c-87cd-1416be49883f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:37.948589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2318bb-0930-470c-87cd-1416be49883f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:37.948872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2318bb-0930-470c-87cd-1416be49883f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:37.949148 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2318bb-0930-470c-87cd-1416be49883f None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:37.949906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2318bb-0930-470c-87cd-1416be49883f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:37.950211 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2318bb-0930-470c-87cd-1416be49883f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:37.951578 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2318bb-0930-470c-87cd-1416be49883f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:37.961019 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 542/1070] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:27:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:37.962329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d573c4-7d93-4e51-bcf1-7a6152e80950 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:37.962620 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d573c4-7d93-4e51-bcf1-7a6152e80950 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:37.962892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d573c4-7d93-4e51-bcf1-7a6152e80950 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:37.965578 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 543/1071] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:37.966881 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d573c4-7d93-4e51-bcf1-7a6152e80950 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:37.976982 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735a619-7420-4b15-869e-10625cb2f2e5 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:38.010963 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdbe14-3957-498d-8e6b-5030c10b0c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:38.011574 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdbe14-3957-498d-8e6b-5030c10b0c61 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:38.011856 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdbe14-3957-498d-8e6b-5030c10b0c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:38.012128 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdbe14-3957-498d-8e6b-5030c10b0c61 None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:38.012923 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdbe14-3957-498d-8e6b-5030c10b0c61 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:38.013227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdbe14-3957-498d-8e6b-5030c10b0c61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:38.014750 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdbe14-3957-498d-8e6b-5030c10b0c61 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:38.032657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 529/1072] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Aug 4 04:27:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:38.033888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39545ed4-cfb1-4ddb-9d0e-6753d3cd3ee3 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:38.034178 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39545ed4-cfb1-4ddb-9d0e-6753d3cd3ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:38.034453 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39545ed4-cfb1-4ddb-9d0e-6753d3cd3ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:38.052145 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-39545ed4-cfb1-4ddb-9d0e-6753d3cd3ee3 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:38.053342 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-39545ed4-cfb1-4ddb-9d0e-6753d3cd3ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:38.074715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d573c4-7d93-4e51-bcf1-7a6152e80950 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:38.092967 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:38.110491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d09844-3ea1-4fec-b146-f326fde6b9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:38.111137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d09844-3ea1-4fec-b146-f326fde6b9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:38.111416 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d09844-3ea1-4fec-b146-f326fde6b9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:38.111696 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d09844-3ea1-4fec-b146-f326fde6b9d8 None admin] PATH_INFO: `/v3/users/455d295339864238844175f3302cd8d0` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:38.112867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d09844-3ea1-4fec-b146-f326fde6b9d8 None admin] RBAC: Authorizing `identity:get_user(user_id=455d295339864238844175f3302cd8d0)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:38.114348 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d09844-3ea1-4fec-b146-f326fde6b9d8 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:38.115946 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 544/1073] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:27:37 2022] GET /identity/v3/users/455d295339864238844175f3302cd8d0 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:38.116695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d09844-3ea1-4fec-b146-f326fde6b9d8 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:38.127328 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 530/1074] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:38.128517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39545ed4-cfb1-4ddb-9d0e-6753d3cd3ee3 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:38.190289 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:38.220585 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d09844-3ea1-4fec-b146-f326fde6b9d8 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:38.232653 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39545ed4-cfb1-4ddb-9d0e-6753d3cd3ee3 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:38.254720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24419ed7-f0b5-4a0f-a708-d59382778e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:38.255408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24419ed7-f0b5-4a0f-a708-d59382778e52 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:38.255683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24419ed7-f0b5-4a0f-a708-d59382778e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:38.255953 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24419ed7-f0b5-4a0f-a708-d59382778e52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:38.266994 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24419ed7-f0b5-4a0f-a708-d59382778e52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:38.268433 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24419ed7-f0b5-4a0f-a708-d59382778e52 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:38.268843 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea57b94-d58c-4e79-abc3-d44650a29180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:38.269485 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea57b94-d58c-4e79-abc3-d44650a29180 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:38.269799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea57b94-d58c-4e79-abc3-d44650a29180 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:38.270080 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea57b94-d58c-4e79-abc3-d44650a29180 None admin] PATH_INFO: `/v3/projects/7f32ef360443455fa75d77dbf975364f` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:38.271308 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea57b94-d58c-4e79-abc3-d44650a29180 None admin] RBAC: Authorizing `identity:get_project(project_id=7f32ef360443455fa75d77dbf975364f)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:38.273163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea57b94-d58c-4e79-abc3-d44650a29180 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:38.275031 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 531/1075] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:27:38 2022] GET /identity/v3/projects/7f32ef360443455fa75d77dbf975364f => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:38.275797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea57b94-d58c-4e79-abc3-d44650a29180 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:38.283357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-24419ed7-f0b5-4a0f-a708-d59382778e52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:38.283357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:38.283357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:38.283357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:38.283357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:38.283357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:38.283357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:38.283357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:38.283357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:38.283357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:38.283357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:38.283357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:38.283357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:27:38.283357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:27:38.283357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:27:38.284400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:27:38.284400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:38.284400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:38.284400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:38.284400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:38.284400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:38.284400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:38.284400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:38.284400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:38.284400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:38.284400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:38.284400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:38.284400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:38.284400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:38.284400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:38.284400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:38.284400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:38.285296 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:27:38.285296 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:27:38.285296 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:38.285296 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:38.285296 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 545/1076] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:38.285296 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24419ed7-f0b5-4a0f-a708-d59382778e52 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:38.325497 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:38.378426 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea57b94-d58c-4e79-abc3-d44650a29180 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:38.386102 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24419ed7-f0b5-4a0f-a708-d59382778e52 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:38.404663 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:38.414221 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ef41c-7385-4efc-9413-449a6d82d58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:38.414844 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ef41c-7385-4efc-9413-449a6d82d58f None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:38.415124 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ef41c-7385-4efc-9413-449a6d82d58f None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:38.415402 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ef41c-7385-4efc-9413-449a6d82d58f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:38.421706 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67aaea9-640c-445a-b7ee-d9cedab1466e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:38.422377 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67aaea9-640c-445a-b7ee-d9cedab1466e None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:38.422655 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67aaea9-640c-445a-b7ee-d9cedab1466e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:38.422926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67aaea9-640c-445a-b7ee-d9cedab1466e None admin] PATH_INFO: `/v3/projects/7f32ef360443455fa75d77dbf975364f/users/455d295339864238844175f3302cd8d0/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:38.424880 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67aaea9-640c-445a-b7ee-d9cedab1466e None admin] RBAC: Authorizing `identity:create_grant(project_id=7f32ef360443455fa75d77dbf975364f, user_id=455d295339864238844175f3302cd8d0, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:38.426334 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67aaea9-640c-445a-b7ee-d9cedab1466e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:38.438978 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 546/1077] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Aug 4 04:27:38 2022] PUT /identity/v3/projects/7f32ef360443455fa75d77dbf975364f/users/455d295339864238844175f3302cd8d0/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:27:38.439773 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67aaea9-640c-445a-b7ee-d9cedab1466e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:38.496721 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ef41c-7385-4efc-9413-449a6d82d58f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:38.498616 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ef41c-7385-4efc-9413-449a6d82d58f None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:38.554989 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:38.559971 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67aaea9-640c-445a-b7ee-d9cedab1466e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:38.606801 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22c848-8302-436c-a542-c9377d4c6dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:38.607512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22c848-8302-436c-a542-c9377d4c6dba None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:38.607792 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22c848-8302-436c-a542-c9377d4c6dba None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:38.608336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22c848-8302-436c-a542-c9377d4c6dba None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:38.609062 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22c848-8302-436c-a542-c9377d4c6dba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:38.609393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22c848-8302-436c-a542-c9377d4c6dba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:38.610961 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22c848-8302-436c-a542-c9377d4c6dba None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:38.624145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 547/1078] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:38.624462 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 532/1079] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:27:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 04:27:38.629891 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ef41c-7385-4efc-9413-449a6d82d58f None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:38.631279 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22c848-8302-436c-a542-c9377d4c6dba None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:38.659628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:38.743378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22c848-8302-436c-a542-c9377d4c6dba None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:38.748513 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ef41c-7385-4efc-9413-449a6d82d58f None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:38.780525 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179187f-0e46-496d-b2f6-920cf79e45ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:38.781168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179187f-0e46-496d-b2f6-920cf79e45ae None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:38.781430 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179187f-0e46-496d-b2f6-920cf79e45ae None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:38.781719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179187f-0e46-496d-b2f6-920cf79e45ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:38.788833 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa78bc-14d4-4731-b570-e6d7a5e1d334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:38.789470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa78bc-14d4-4731-b570-e6d7a5e1d334 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:38.789900 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa78bc-14d4-4731-b570-e6d7a5e1d334 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:38.790207 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa78bc-14d4-4731-b570-e6d7a5e1d334 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:38.791984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179187f-0e46-496d-b2f6-920cf79e45ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:38.793437 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179187f-0e46-496d-b2f6-920cf79e45ae None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:38.798079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaa78bc-14d4-4731-b570-e6d7a5e1d334 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:38.799570 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaa78bc-14d4-4731-b570-e6d7a5e1d334 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:38.808440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-6179187f-0e46-496d-b2f6-920cf79e45ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:38.808440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:38.808440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:38.808440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:38.808440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:38.808440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:38.808440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:38.808440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:38.808440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:38.808440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:38.808440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:38.808440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:38.808440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:27:38.808440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:27:38.808440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:27:38.809517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:27:38.809517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:38.809517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:38.809517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:38.809517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:38.809517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:38.809517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:38.809517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:38.809517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:38.809517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:38.809517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:38.809517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:38.809517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:38.809517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:38.809517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:38.809517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:38.809517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:38.810925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:27:38.810925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:27:38.810925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:38.810925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:38.813921 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 548/1080] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:38.814062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-1aaa78bc-14d4-4731-b570-e6d7a5e1d334 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:38.814062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:38.814062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:38.814062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:38.814062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:38.814062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:38.814062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:38.814062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:38.814062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:38.814062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:38.814062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:38.814062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:38.814062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:27:38.814062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:27:38.814062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:27:38.815211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:27:38.815211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:38.815211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:38.815211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:38.815211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:38.815211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:38.815211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:38.815211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:38.815211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:38.815211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:38.815211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:38.815211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:38.815211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:38.815211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:38.815211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:38.815211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:38.815211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:38.816612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:27:38.816612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:27:38.816612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:27:38.816612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:27:38.816612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:27:38.816612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:27:38.816612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:38.816612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:38.817201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 533/1081] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:27:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:38.818280 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa78bc-14d4-4731-b570-e6d7a5e1d334 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:38.822263 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179187f-0e46-496d-b2f6-920cf79e45ae None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:38.836239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:38.913426 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:38.933794 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa78bc-14d4-4731-b570-e6d7a5e1d334 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:38.934967 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179187f-0e46-496d-b2f6-920cf79e45ae None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:38.971318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7b8cd-aa79-4305-b9f5-668233fdbc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:38.972054 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7b8cd-aa79-4305-b9f5-668233fdbc63 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:38.972333 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7b8cd-aa79-4305-b9f5-668233fdbc63 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:38.972600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7b8cd-aa79-4305-b9f5-668233fdbc63 None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:38.973197 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb275a-51d1-4a7e-899f-c127f71cf5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:38.973328 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf7b8cd-aa79-4305-b9f5-668233fdbc63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:38.973598 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf7b8cd-aa79-4305-b9f5-668233fdbc63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:38.973865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb275a-51d1-4a7e-899f-c127f71cf5bc None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:38.974153 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb275a-51d1-4a7e-899f-c127f71cf5bc None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:38.974546 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb275a-51d1-4a7e-899f-c127f71cf5bc None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:38.974948 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf7b8cd-aa79-4305-b9f5-668233fdbc63 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:38.975395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb275a-51d1-4a7e-899f-c127f71cf5bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:38.975710 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb275a-51d1-4a7e-899f-c127f71cf5bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:38.977121 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb275a-51d1-4a7e-899f-c127f71cf5bc None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:38.987333 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 534/1082] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:38.994710 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 549/1083] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:27:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:38.996019 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7b8cd-aa79-4305-b9f5-668233fdbc63 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:39.110348 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7b8cd-aa79-4305-b9f5-668233fdbc63 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:39.148026 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b73182-251e-459a-8a64-0a7beb962d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:39.148709 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b73182-251e-459a-8a64-0a7beb962d2f None admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:39.148995 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b73182-251e-459a-8a64-0a7beb962d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:39.149271 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b73182-251e-459a-8a64-0a7beb962d2f None admin] PATH_INFO: `/v3/projects/f37a5f3140ec431da8ea14568654114a/users/caa2901451aa4554bdfcac1bddbc620a/roles/3747dec438cd4349bd9e402694b88574` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:39.153090 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b73182-251e-459a-8a64-0a7beb962d2f None admin] RBAC: Authorizing `identity:create_grant(project_id=f37a5f3140ec431da8ea14568654114a, user_id=caa2901451aa4554bdfcac1bddbc620a, role_id=3747dec438cd4349bd9e402694b88574)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:39.154565 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b73182-251e-459a-8a64-0a7beb962d2f None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:39.171636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 535/1084] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Aug 4 04:27:38 2022] PUT /identity/v3/projects/f37a5f3140ec431da8ea14568654114a/users/caa2901451aa4554bdfcac1bddbc620a/roles/3747dec438cd4349bd9e402694b88574 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:27:40.447676 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebd0a6-edf8-4533-a148-9d2bbb7faf72 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:40.448211 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebd0a6-edf8-4533-a148-9d2bbb7faf72 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:40.448211 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebd0a6-edf8-4533-a148-9d2bbb7faf72 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:40.449760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 550/1085] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:40.455727 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a37704-c406-4931-9453-a8e98e3bf3b5 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:40.456118 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a37704-c406-4931-9453-a8e98e3bf3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:40.456396 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a37704-c406-4931-9453-a8e98e3bf3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:40.470123 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:40.475684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-c9a37704-c406-4931-9453-a8e98e3bf3b5 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:40.477412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c9a37704-c406-4931-9453-a8e98e3bf3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:40.547590 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 536/1086] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:40.569914 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebd0a6-edf8-4533-a148-9d2bbb7faf72 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:40.638776 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:40.671484 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebd0a6-edf8-4533-a148-9d2bbb7faf72 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:40.705526 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1b391-ce55-4d59-a1b2-3ed93ddb2a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:40.706115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1b391-ce55-4d59-a1b2-3ed93ddb2a9e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:40.706395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1b391-ce55-4d59-a1b2-3ed93ddb2a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:40.706664 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1b391-ce55-4d59-a1b2-3ed93ddb2a9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:40.716695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1b391-ce55-4d59-a1b2-3ed93ddb2a9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:27:40.722368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-c5b1b391-ce55-4d59-a1b2-3ed93ddb2a9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:27:40.722368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:40.722368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:40.722368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:40.722368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:40.722368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:40.722368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:40.722368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:40.722368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:40.722368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:40.722368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:40.722368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:40.722368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:27:40.722368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:27:40.722368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:27:40.722368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:27:40.723310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:27:40.723310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:27:40.723310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:27:40.723310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:27:40.723310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:40.723310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:40.723310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:40.723310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:40.723310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:40.723310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:40.723310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:40.723310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:40.723310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:40.723310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:40.723310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:40.723310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:40.724361 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:40.724361 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:40.724361 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:40.724361 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:40.724361 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:27:40.724361 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:27:40.724361 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:27:40.724361 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:27:40.724361 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:27:40.724361 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:27:40.724361 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:27:40.724361 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:40.724361 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 551/1087] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:27:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:40.729506 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a37704-c406-4931-9453-a8e98e3bf3b5 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:40.829086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a37704-c406-4931-9453-a8e98e3bf3b5 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:40.862125 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a61ee-840d-4906-b344-70c422005f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:40.862729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a61ee-840d-4906-b344-70c422005f77 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:40.863006 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a61ee-840d-4906-b344-70c422005f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:40.863273 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a61ee-840d-4906-b344-70c422005f77 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:40.863975 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a61ee-840d-4906-b344-70c422005f77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:40.864272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a61ee-840d-4906-b344-70c422005f77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:40.865636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a61ee-840d-4906-b344-70c422005f77 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:40.867752 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:40.874674 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 537/1088] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:27:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:40.880369 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1b391-ce55-4d59-a1b2-3ed93ddb2a9e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:40.979683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1b391-ce55-4d59-a1b2-3ed93ddb2a9e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:41.007569 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:41.013357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79fcc3c-fd0f-4b34-9afb-2d80c5784488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:41.014082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fcc3c-fd0f-4b34-9afb-2d80c5784488 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:41.014853 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fcc3c-fd0f-4b34-9afb-2d80c5784488 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:41.015291 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fcc3c-fd0f-4b34-9afb-2d80c5784488 None admin] PATH_INFO: `/v3/users/455d295339864238844175f3302cd8d0` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:41.016482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79fcc3c-fd0f-4b34-9afb-2d80c5784488 None admin] RBAC: Authorizing `identity:get_user(user_id=455d295339864238844175f3302cd8d0)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:41.017938 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79fcc3c-fd0f-4b34-9afb-2d80c5784488 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:41.019540 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 552/1089] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:27:40 2022] GET /identity/v3/users/455d295339864238844175f3302cd8d0 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:41.025237 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a61ee-840d-4906-b344-70c422005f77 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:41.126728 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a61ee-840d-4906-b344-70c422005f77 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:41.159965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cf262-595f-457d-bc69-886b609326e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:41.160573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cf262-595f-457d-bc69-886b609326e1 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:41.160849 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cf262-595f-457d-bc69-886b609326e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:41.161116 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cf262-595f-457d-bc69-886b609326e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:41.199740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318cc07b-a854-44af-a45f-66a88df2e5c4 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:41.200027 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318cc07b-a854-44af-a45f-66a88df2e5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:41.200341 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318cc07b-a854-44af-a45f-66a88df2e5c4 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:41.201996 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 553/1090] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:27:41.207018 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476864a-a44f-4576-953b-41f54b3f2ab6 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:41.207297 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476864a-a44f-4576-953b-41f54b3f2ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:41.207570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476864a-a44f-4576-953b-41f54b3f2ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:41.225539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-e476864a-a44f-4576-953b-41f54b3f2ab6 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:41.226629 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:41.226846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e476864a-a44f-4576-953b-41f54b3f2ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:41.230814 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7cf262-595f-457d-bc69-886b609326e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:41.232232 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7cf262-595f-457d-bc69-886b609326e1 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:41.294537 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 554/1091] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 04:27:41.318385 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e476864a-a44f-4576-953b-41f54b3f2ab6 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:41.330565 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 538/1092] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:27:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 04:27:41.336470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cf262-595f-457d-bc69-886b609326e1 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:41.422346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e476864a-a44f-4576-953b-41f54b3f2ab6 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:41.432008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:41.436644 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cf262-595f-457d-bc69-886b609326e1 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:41.448317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:41.456443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa33d8-c4ef-4f6f-9ae3-87a47ea96155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:41.457076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa33d8-c4ef-4f6f-9ae3-87a47ea96155 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:41.457354 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa33d8-c4ef-4f6f-9ae3-87a47ea96155 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:41.457653 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa33d8-c4ef-4f6f-9ae3-87a47ea96155 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:41.467552 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa33d8-c4ef-4f6f-9ae3-87a47ea96155 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:27:41.472376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-8daa33d8-c4ef-4f6f-9ae3-87a47ea96155 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 04:27:41.472376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:41.472376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:41.472376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:41.472376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:41.472376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:41.472376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:41.472376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:41.472376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:41.472376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:41.472376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:41.472376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:41.472376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:27:41.472376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:27:41.472376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:27:41.472376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:27:41.473288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:27:41.473288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:27:41.473288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:27:41.473288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:27:41.473288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:41.473288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:41.473288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:41.473288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:41.473288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:41.473288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:41.473288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:41.473288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:41.473288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:41.473288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:41.473288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:41.473288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:41.474159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:41.474159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:41.474159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:41.474159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:41.474159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:27:41.474159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:27:41.474159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:27:41.474159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:27:41.474159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:27:41.474159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:27:41.474159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 04:27:41.474159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:41.474821 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4c7c3-2d84-408b-abca-9c1b333f7e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:41.474821 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4c7c3-2d84-408b-abca-9c1b333f7e61 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:41.474821 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4c7c3-2d84-408b-abca-9c1b333f7e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:41.475341 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 555/1093] 10.222.0.47 () {56 vars in 1120 bytes} [Thu Aug 4 04:27:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:41.475412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4c7c3-2d84-408b-abca-9c1b333f7e61 None admin] PATH_INFO: `/v3/projects/7f32ef360443455fa75d77dbf975364f` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:41.475412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4c7c3-2d84-408b-abca-9c1b333f7e61 None admin] RBAC: Authorizing `identity:get_project(project_id=7f32ef360443455fa75d77dbf975364f)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:41.476278 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4c7c3-2d84-408b-abca-9c1b333f7e61 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:41.483277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 539/1094] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:27:41 2022] GET /identity/v3/projects/7f32ef360443455fa75d77dbf975364f => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:41.483405 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa33d8-c4ef-4f6f-9ae3-87a47ea96155 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:41.484173 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4c7c3-2d84-408b-abca-9c1b333f7e61 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:41.586003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa33d8-c4ef-4f6f-9ae3-87a47ea96155 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:41.587799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4c7c3-2d84-408b-abca-9c1b333f7e61 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:41.620358 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01caf843-ddff-44af-862e-1b4271f90ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:41.620965 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01caf843-ddff-44af-862e-1b4271f90ddc None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:41.621247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01caf843-ddff-44af-862e-1b4271f90ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:41.621518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01caf843-ddff-44af-862e-1b4271f90ddc None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:41.622145 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1beacc-be7c-4a59-bd71-ddef448452a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:41.622259 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01caf843-ddff-44af-862e-1b4271f90ddc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:41.622550 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01caf843-ddff-44af-862e-1b4271f90ddc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:41.622755 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1beacc-be7c-4a59-bd71-ddef448452a2 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:41.623060 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1beacc-be7c-4a59-bd71-ddef448452a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:41.623326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1beacc-be7c-4a59-bd71-ddef448452a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:41.623945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01caf843-ddff-44af-862e-1b4271f90ddc None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:41.624197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1beacc-be7c-4a59-bd71-ddef448452a2 None admin] RBAC: Adding query filter params (role.id=4a6517e5890a49718303007dac0756c7, scope.project.id=7f32ef360443455fa75d77dbf975364f, user.id=455d295339864238844175f3302cd8d0) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:41.624509 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1beacc-be7c-4a59-bd71-ddef448452a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:41.625943 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1beacc-be7c-4a59-bd71-ddef448452a2 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:41.631861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 540/1095] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:27:41 2022] GET /identity/v3/role_assignments?user.id=455d295339864238844175f3302cd8d0&scope.project.id=7f32ef360443455fa75d77dbf975364f&role.id=4a6517e5890a49718303007dac0756c7 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:41.631983 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 556/1096] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Aug 4 04:27:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:41.637615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1beacc-be7c-4a59-bd71-ddef448452a2 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:41.677419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:41.753759 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1beacc-be7c-4a59-bd71-ddef448452a2 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:41.789427 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4374e17-35dc-4a25-b2a1-d828d09b0c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:41.790108 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4374e17-35dc-4a25-b2a1-d828d09b0c97 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:41.790385 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4374e17-35dc-4a25-b2a1-d828d09b0c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:41.790659 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4374e17-35dc-4a25-b2a1-d828d09b0c97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:41.802195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4374e17-35dc-4a25-b2a1-d828d09b0c97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:41.803639 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4374e17-35dc-4a25-b2a1-d828d09b0c97 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:41.819866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-a4374e17-35dc-4a25-b2a1-d828d09b0c97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:41.819866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:41.819866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:41.819866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:41.819866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:41.819866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:41.819866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:41.819866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:41.819866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:41.819866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:41.819866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:41.819866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:41.819866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:27:41.819866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:27:41.819866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:27:41.820797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:27:41.820797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:41.820797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:41.820797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:41.820797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:41.820797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:41.820797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:41.820797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:41.820797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:41.820797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:41.820797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:41.820797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:41.820797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:41.820797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:41.820797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:41.820797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:41.820797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:41.821727 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:27:41.821727 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:27:41.821727 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:41.821727 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:41.821727 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 541/1097] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:41.827568 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01caf843-ddff-44af-862e-1b4271f90ddc None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:41.879136 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:41.931521 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01caf843-ddff-44af-862e-1b4271f90ddc None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:41.965285 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc5b32-b6fc-463e-a7cb-e3b2a8e5f8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:41.965935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc5b32-b6fc-463e-a7cb-e3b2a8e5f8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:41.966219 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc5b32-b6fc-463e-a7cb-e3b2a8e5f8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:41.966494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc5b32-b6fc-463e-a7cb-e3b2a8e5f8b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:41.967706 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc5b32-b6fc-463e-a7cb-e3b2a8e5f8b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:41.968011 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc5b32-b6fc-463e-a7cb-e3b2a8e5f8b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:41.969404 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc5b32-b6fc-463e-a7cb-e3b2a8e5f8b2 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:41.981766 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 557/1098] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:41.987535 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4374e17-35dc-4a25-b2a1-d828d09b0c97 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:42.070181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:42.089048 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4374e17-35dc-4a25-b2a1-d828d09b0c97 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:42.122213 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b01c68-722d-456b-9033-b283f488aa53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:42.122845 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b01c68-722d-456b-9033-b283f488aa53 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:42.123126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b01c68-722d-456b-9033-b283f488aa53 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:42.123930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b01c68-722d-456b-9033-b283f488aa53 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:42.131852 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b01c68-722d-456b-9033-b283f488aa53 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:42.133290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b01c68-722d-456b-9033-b283f488aa53 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:42.145410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-78b01c68-722d-456b-9033-b283f488aa53 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 04:27:42.145410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:42.145410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:42.145410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:42.145410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:42.145410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:42.145410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:42.145410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:42.145410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:42.145410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:42.145410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:42.145410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:42.145410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:27:42.145410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:27:42.145410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:27:42.145410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:27:42.147177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:42.147177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:42.147177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:27:42.147177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:42.147177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:27:42.147177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:27:42.147177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:42.147177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:42.147177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:42.147177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:42.147177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:42.147177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:42.147177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:42.147177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:42.147177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:42.147177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:42.147177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:42.147177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:42.148162 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:42.148162 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:42.148162 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:27:42.148162 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:27:42.148162 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:27:42.148162 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:27:42.148162 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:27:42.148162 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:27:42.148162 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 04:27:42.148162 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:27:42.148162 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 542/1099] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Aug 4 04:27:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:42.152591 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc5b32-b6fc-463e-a7cb-e3b2a8e5f8b2 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:42.251488 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc5b32-b6fc-463e-a7cb-e3b2a8e5f8b2 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:42.262170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:42.286415 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9c8e0-2b56-4b2f-bbb5-b248feb94202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:42.287029 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9c8e0-2b56-4b2f-bbb5-b248feb94202 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:42.287307 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9c8e0-2b56-4b2f-bbb5-b248feb94202 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:42.287576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9c8e0-2b56-4b2f-bbb5-b248feb94202 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:42.288358 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce9c8e0-2b56-4b2f-bbb5-b248feb94202 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:42.288661 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce9c8e0-2b56-4b2f-bbb5-b248feb94202 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:42.290118 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce9c8e0-2b56-4b2f-bbb5-b248feb94202 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:42.307897 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 558/1100] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Aug 4 04:27:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:42.314296 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b01c68-722d-456b-9033-b283f488aa53 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:42.413321 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b01c68-722d-456b-9033-b283f488aa53 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:42.448894 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93701ed-da92-4403-931d-207c2b878a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:42.449498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93701ed-da92-4403-931d-207c2b878a63 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:42.449799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93701ed-da92-4403-931d-207c2b878a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:42.450065 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93701ed-da92-4403-931d-207c2b878a63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:42.461814 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:42.519199 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93701ed-da92-4403-931d-207c2b878a63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:42.520648 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93701ed-da92-4403-931d-207c2b878a63 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:42.619694 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 543/1101] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:27:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 04:27:42.625697 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9c8e0-2b56-4b2f-bbb5-b248feb94202 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:42.723545 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9c8e0-2b56-4b2f-bbb5-b248feb94202 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:42.757399 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87ec73-769c-4b1a-91bc-05061fb5ec89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:42.758112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87ec73-769c-4b1a-91bc-05061fb5ec89 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:42.758391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87ec73-769c-4b1a-91bc-05061fb5ec89 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:42.758660 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87ec73-769c-4b1a-91bc-05061fb5ec89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:42.769500 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87ec73-769c-4b1a-91bc-05061fb5ec89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:42.770988 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87ec73-769c-4b1a-91bc-05061fb5ec89 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:42.785958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-4d87ec73-769c-4b1a-91bc-05061fb5ec89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:42.785958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:42.785958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:42.785958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:42.785958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:42.785958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:42.785958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:42.785958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:42.785958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:42.785958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:42.785958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:42.785958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:42.785958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:27:42.785958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:27:42.785958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:27:42.786885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:27:42.786885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:42.786885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:42.786885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:42.786885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:42.786885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:42.786885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:42.786885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:42.786885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:42.786885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:42.786885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:42.786885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:42.786885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:42.786885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:42.786885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:42.786885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:42.786885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:42.787900 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:27:42.787900 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:27:42.787900 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:27:42.787900 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:42.787900 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 559/1102] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:27:42.794694 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93701ed-da92-4403-931d-207c2b878a63 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:42.840063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:42.896146 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93701ed-da92-4403-931d-207c2b878a63 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:42.930403 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1fa74-515b-407b-ab1c-623d2d641f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:42.931100 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1fa74-515b-407b-ab1c-623d2d641f9f None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:42.931376 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1fa74-515b-407b-ab1c-623d2d641f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:42.931645 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1fa74-515b-407b-ab1c-623d2d641f9f None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:42.932327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1fa74-515b-407b-ab1c-623d2d641f9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:42.932628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1fa74-515b-407b-ab1c-623d2d641f9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:42.934006 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1fa74-515b-407b-ab1c-623d2d641f9f None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:42.945848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 544/1103] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:42.951462 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87ec73-769c-4b1a-91bc-05061fb5ec89 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:42.972930 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:43.051273 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87ec73-769c-4b1a-91bc-05061fb5ec89 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:43.084608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322834e-5005-4249-97b3-e2ce385d48ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:43.085237 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322834e-5005-4249-97b3-e2ce385d48ad None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:43.085514 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322834e-5005-4249-97b3-e2ce385d48ad None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:43.085808 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322834e-5005-4249-97b3-e2ce385d48ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:43.092407 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322834e-5005-4249-97b3-e2ce385d48ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:43.093862 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322834e-5005-4249-97b3-e2ce385d48ad None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:43.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-9322834e-5005-4249-97b3-e2ce385d48ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:43.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:27:43.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:27:43.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:27:43.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:27:43.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:27:43.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:27:43.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:27:43.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:27:43.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:27:43.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:27:43.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:27:43.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:27:43.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:27:43.104925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:27:43.105839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:27:43.105839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:27:43.105839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:27:43.105839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:27:43.105839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:27:43.105839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:27:43.105839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:27:43.105839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:27:43.105839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:27:43.105839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:27:43.105839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:27:43.105839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:27:43.105839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:27:43.105839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:27:43.105839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:27:43.105839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:27:43.105839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:27:43.106920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:27:43.106920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:27:43.106920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:27:43.106920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:27:43.106920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:27:43.106920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:27:43.106920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:27:43.106920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:27:43.106920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 560/1104] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:27:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:27:43.112603 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1fa74-515b-407b-ab1c-623d2d641f9f None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:43.211797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1fa74-515b-407b-ab1c-623d2d641f9f None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:43.213323 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:43.245875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7381b-f8ba-4ad3-9982-69e0ce384779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:43.246483 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7381b-f8ba-4ad3-9982-69e0ce384779 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:43.246758 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7381b-f8ba-4ad3-9982-69e0ce384779 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:43.247025 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7381b-f8ba-4ad3-9982-69e0ce384779 None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:43.247759 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b7381b-f8ba-4ad3-9982-69e0ce384779 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:43.248066 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b7381b-f8ba-4ad3-9982-69e0ce384779 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:43.250303 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b7381b-f8ba-4ad3-9982-69e0ce384779 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:43.261822 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 545/1105] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:27:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:43.268331 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322834e-5005-4249-97b3-e2ce385d48ad None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:43.348795 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:43.367606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322834e-5005-4249-97b3-e2ce385d48ad None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:43.401003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18dabde-2c98-4c6c-9867-8f92aabe33c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:43.401641 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dabde-2c98-4c6c-9867-8f92aabe33c6 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:43.401920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dabde-2c98-4c6c-9867-8f92aabe33c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:43.402186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dabde-2c98-4c6c-9867-8f92aabe33c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:43.402970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18dabde-2c98-4c6c-9867-8f92aabe33c6 None admin] RBAC: Adding query filter params (role.id=3747dec438cd4349bd9e402694b88574, scope.project.id=f37a5f3140ec431da8ea14568654114a, user.id=caa2901451aa4554bdfcac1bddbc620a) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:43.403274 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18dabde-2c98-4c6c-9867-8f92aabe33c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:43.405261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18dabde-2c98-4c6c-9867-8f92aabe33c6 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:43.411247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 561/1106] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:27:43 2022] GET /identity/v3/role_assignments?user.id=caa2901451aa4554bdfcac1bddbc620a&scope.project.id=f37a5f3140ec431da8ea14568654114a&role.id=3747dec438cd4349bd9e402694b88574 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:43.740935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5570f2-6c5f-480c-a9e9-324a9003acb8 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:43.741445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5570f2-6c5f-480c-a9e9-324a9003acb8 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:43.741445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5570f2-6c5f-480c-a9e9-324a9003acb8 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:43.742773 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 546/1107] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:43.748456 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbada0a-679d-48b8-a102-4f2458b921ec None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:43.748886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbada0a-679d-48b8-a102-4f2458b921ec None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:43.748996 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbada0a-679d-48b8-a102-4f2458b921ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:43.767552 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-cfbada0a-679d-48b8-a102-4f2458b921ec None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:43.768738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-cfbada0a-679d-48b8-a102-4f2458b921ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:43.838133 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 562/1108] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:43.863260 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5570f2-6c5f-480c-a9e9-324a9003acb8 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:43.968419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5570f2-6c5f-480c-a9e9-324a9003acb8 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:44.002311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a37ec-e4da-494d-aefb-e9c284fe484c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:44.002874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a37ec-e4da-494d-aefb-e9c284fe484c None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:44.003150 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a37ec-e4da-494d-aefb-e9c284fe484c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:44.003445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a37ec-e4da-494d-aefb-e9c284fe484c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:44.004602 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a37ec-e4da-494d-aefb-e9c284fe484c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:44.006118 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a37ec-e4da-494d-aefb-e9c284fe484c None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:44.007696 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 547/1109] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:44.013855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbada0a-679d-48b8-a102-4f2458b921ec None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:44.091714 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:44.118174 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbada0a-679d-48b8-a102-4f2458b921ec None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:44.153860 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65647ccd-89da-4343-971e-0430304a1972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:44.154488 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65647ccd-89da-4343-971e-0430304a1972 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:44.154774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65647ccd-89da-4343-971e-0430304a1972 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:44.155047 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65647ccd-89da-4343-971e-0430304a1972 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:44.155736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65647ccd-89da-4343-971e-0430304a1972 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:44.157173 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65647ccd-89da-4343-971e-0430304a1972 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:44.182241 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 563/1110] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Aug 4 04:27:44 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:27:46.248629 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319eab5-360c-41cd-b52d-7fdebb37f96a None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:46.249296 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319eab5-360c-41cd-b52d-7fdebb37f96a None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:46.249296 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319eab5-360c-41cd-b52d-7fdebb37f96a None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:46.250535 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 548/1111] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:46.256532 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ce4b3-a869-4224-b600-aab6efd8be45 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:46.256982 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ce4b3-a869-4224-b600-aab6efd8be45 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:46.257091 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ce4b3-a869-4224-b600-aab6efd8be45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:46.274873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-366ce4b3-a869-4224-b600-aab6efd8be45 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:46.276033 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-366ce4b3-a869-4224-b600-aab6efd8be45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:46.341012 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 564/1112] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:46.364014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319eab5-360c-41cd-b52d-7fdebb37f96a None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:46.394948 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:46.466344 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319eab5-360c-41cd-b52d-7fdebb37f96a None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:46.500540 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5827ee-7f0e-4dd7-a160-d261b7d923d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:46.501105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5827ee-7f0e-4dd7-a160-d261b7d923d0 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:46.501384 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5827ee-7f0e-4dd7-a160-d261b7d923d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:46.501688 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5827ee-7f0e-4dd7-a160-d261b7d923d0 None admin] PATH_INFO: `/v3/roles/7a855ec41b944458821d76509b481268` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:46.503139 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5827ee-7f0e-4dd7-a160-d261b7d923d0 None admin] RBAC: Authorizing `identity:get_role(role_id=7a855ec41b944458821d76509b481268)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:46.504490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5827ee-7f0e-4dd7-a160-d261b7d923d0 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:46.505854 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 549/1113] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:27:46 2022] GET /identity/v3/roles/7a855ec41b944458821d76509b481268 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:46.511812 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ce4b3-a869-4224-b600-aab6efd8be45 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:46.568903 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:46.609070 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ce4b3-a869-4224-b600-aab6efd8be45 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:46.642504 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067554a3-dfa8-42bb-97dc-5c0a2052b73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:46.643137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067554a3-dfa8-42bb-97dc-5c0a2052b73d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:46.643421 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067554a3-dfa8-42bb-97dc-5c0a2052b73d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:46.643696 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067554a3-dfa8-42bb-97dc-5c0a2052b73d None admin] PATH_INFO: `/v3/users/58d00046b0324af39b03b6171765b9b5` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:46.657141 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067554a3-dfa8-42bb-97dc-5c0a2052b73d None admin] RBAC: Authorizing `identity:get_user(user_id=58d00046b0324af39b03b6171765b9b5)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:46.658596 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067554a3-dfa8-42bb-97dc-5c0a2052b73d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:46.660185 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 565/1114] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:27:46 2022] GET /identity/v3/users/58d00046b0324af39b03b6171765b9b5 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:46.665951 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5827ee-7f0e-4dd7-a160-d261b7d923d0 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:46.763636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:46.768012 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5827ee-7f0e-4dd7-a160-d261b7d923d0 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:46.804357 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145e3aa-158e-406d-9b95-0734cdaea722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:46.804966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145e3aa-158e-406d-9b95-0734cdaea722 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:46.805248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145e3aa-158e-406d-9b95-0734cdaea722 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:46.805523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145e3aa-158e-406d-9b95-0734cdaea722 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:46.871366 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2145e3aa-158e-406d-9b95-0734cdaea722 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:46.872791 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2145e3aa-158e-406d-9b95-0734cdaea722 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:46.969794 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:46.975727 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 550/1115] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:27:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 04:27:46.980476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067554a3-dfa8-42bb-97dc-5c0a2052b73d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:47.086025 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067554a3-dfa8-42bb-97dc-5c0a2052b73d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:47.106977 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:47.129510 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00ee9a-d0d5-47e0-9f3f-9b68408e611f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:47.130179 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00ee9a-d0d5-47e0-9f3f-9b68408e611f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:47.130457 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00ee9a-d0d5-47e0-9f3f-9b68408e611f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:47.130730 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00ee9a-d0d5-47e0-9f3f-9b68408e611f None admin] PATH_INFO: `/v3/projects/7f32ef360443455fa75d77dbf975364f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:47.131896 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00ee9a-d0d5-47e0-9f3f-9b68408e611f None admin] RBAC: Authorizing `identity:get_project(project_id=7f32ef360443455fa75d77dbf975364f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:47.133724 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00ee9a-d0d5-47e0-9f3f-9b68408e611f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:47.136573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 566/1116] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:27:46 2022] GET /identity/v3/projects/7f32ef360443455fa75d77dbf975364f => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:47.145599 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145e3aa-158e-406d-9b95-0734cdaea722 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:47.256794 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145e3aa-158e-406d-9b95-0734cdaea722 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:47.294309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f3bd0-83d1-4d89-8934-616a8c1f7e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:47.295474 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f3bd0-83d1-4d89-8934-616a8c1f7e33 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:47.295763 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f3bd0-83d1-4d89-8934-616a8c1f7e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:47.296137 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f3bd0-83d1-4d89-8934-616a8c1f7e33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:47.297479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f3bd0-83d1-4d89-8934-616a8c1f7e33 None admin] RBAC: Adding query filter params (role.id=7a855ec41b944458821d76509b481268, scope.project.id=7f32ef360443455fa75d77dbf975364f, user.id=58d00046b0324af39b03b6171765b9b5) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:47.297813 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f3bd0-83d1-4d89-8934-616a8c1f7e33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:47.299252 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f3bd0-83d1-4d89-8934-616a8c1f7e33 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:47.305092 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 551/1117] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:27:47 2022] GET /identity/v3/role_assignments?user.id=58d00046b0324af39b03b6171765b9b5&scope.project.id=7f32ef360443455fa75d77dbf975364f&role.id=7a855ec41b944458821d76509b481268 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:49.336388 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e30c47-7fad-46c0-b6a3-3b10d500093a None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:49.336917 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e30c47-7fad-46c0-b6a3-3b10d500093a None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:49.336917 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e30c47-7fad-46c0-b6a3-3b10d500093a None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:49.338309 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 567/1118] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:49.344170 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda1ca9-8ff6-428e-b98d-07a914c7029c None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:49.344618 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda1ca9-8ff6-428e-b98d-07a914c7029c None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:49.344718 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda1ca9-8ff6-428e-b98d-07a914c7029c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:49.363450 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-8cda1ca9-8ff6-428e-b98d-07a914c7029c None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:49.364631 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-8cda1ca9-8ff6-428e-b98d-07a914c7029c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:49.418598 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:49.430221 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 552/1119] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 04:27:49.453037 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e30c47-7fad-46c0-b6a3-3b10d500093a None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:49.552331 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e30c47-7fad-46c0-b6a3-3b10d500093a None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:49.584700 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8dd5d-e40d-444d-a32b-63b5dd925788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:49.585327 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8dd5d-e40d-444d-a32b-63b5dd925788 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:49.585634 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8dd5d-e40d-444d-a32b-63b5dd925788 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:49.585911 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8dd5d-e40d-444d-a32b-63b5dd925788 None admin] PATH_INFO: `/v3/roles/7a855ec41b944458821d76509b481268` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:49.587396 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a8dd5d-e40d-444d-a32b-63b5dd925788 None admin] RBAC: Authorizing `identity:get_role(role_id=7a855ec41b944458821d76509b481268)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:49.588717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a8dd5d-e40d-444d-a32b-63b5dd925788 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:49.590060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 568/1120] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:27:49 2022] GET /identity/v3/roles/7a855ec41b944458821d76509b481268 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:49.596689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda1ca9-8ff6-428e-b98d-07a914c7029c None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:49.696287 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda1ca9-8ff6-428e-b98d-07a914c7029c None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:49.730310 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a9c54-495e-4aba-84ec-aedf9e6b2444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:49.730933 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a9c54-495e-4aba-84ec-aedf9e6b2444 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:49.731213 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a9c54-495e-4aba-84ec-aedf9e6b2444 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:49.731484 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a9c54-495e-4aba-84ec-aedf9e6b2444 None admin] PATH_INFO: `/v3/users/58d00046b0324af39b03b6171765b9b5` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:49.732981 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a9c54-495e-4aba-84ec-aedf9e6b2444 None admin] RBAC: Authorizing `identity:get_user(user_id=58d00046b0324af39b03b6171765b9b5)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:49.734428 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a9c54-495e-4aba-84ec-aedf9e6b2444 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:49.736018 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 553/1121] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:27:49 2022] GET /identity/v3/users/58d00046b0324af39b03b6171765b9b5 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:49.741765 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8dd5d-e40d-444d-a32b-63b5dd925788 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:49.797878 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:49.841369 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8dd5d-e40d-444d-a32b-63b5dd925788 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:49.874312 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141083c-98a8-4ccd-a17d-37c079e813d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:49.874931 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141083c-98a8-4ccd-a17d-37c079e813d2 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:49.875211 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141083c-98a8-4ccd-a17d-37c079e813d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:49.875494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141083c-98a8-4ccd-a17d-37c079e813d2 None admin] PATH_INFO: `/v3/projects/7f32ef360443455fa75d77dbf975364f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:49.876635 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f141083c-98a8-4ccd-a17d-37c079e813d2 None admin] RBAC: Authorizing `identity:get_project(project_id=7f32ef360443455fa75d77dbf975364f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:49.878212 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f141083c-98a8-4ccd-a17d-37c079e813d2 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:49.879842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 569/1122] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:27:49 2022] GET /identity/v3/projects/7f32ef360443455fa75d77dbf975364f => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:49.885961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a9c54-495e-4aba-84ec-aedf9e6b2444 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:49.945237 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:49.987510 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a9c54-495e-4aba-84ec-aedf9e6b2444 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:50.021636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee98ca5-2be0-4637-9e33-900c5b287273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:50.022289 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee98ca5-2be0-4637-9e33-900c5b287273 None admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:50.022568 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee98ca5-2be0-4637-9e33-900c5b287273 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:50.022841 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee98ca5-2be0-4637-9e33-900c5b287273 None admin] PATH_INFO: `/v3/projects/7f32ef360443455fa75d77dbf975364f/users/58d00046b0324af39b03b6171765b9b5/roles/7a855ec41b944458821d76509b481268` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:50.024979 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee98ca5-2be0-4637-9e33-900c5b287273 None admin] RBAC: Authorizing `identity:create_grant(project_id=7f32ef360443455fa75d77dbf975364f, user_id=58d00046b0324af39b03b6171765b9b5, role_id=7a855ec41b944458821d76509b481268)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:50.026417 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee98ca5-2be0-4637-9e33-900c5b287273 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:50.039223 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 554/1123] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Aug 4 04:27:49 2022] PUT /identity/v3/projects/7f32ef360443455fa75d77dbf975364f/users/58d00046b0324af39b03b6171765b9b5/roles/7a855ec41b944458821d76509b481268 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:27:52.011713 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b1876-3055-4213-9d2c-ef3249045e1b None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:52.012253 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b1876-3055-4213-9d2c-ef3249045e1b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:52.012253 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b1876-3055-4213-9d2c-ef3249045e1b None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:52.013736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 570/1124] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:52.019296 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf027a7d-cc06-4709-8545-416676302cc7 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:52.019716 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf027a7d-cc06-4709-8545-416676302cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:52.019837 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf027a7d-cc06-4709-8545-416676302cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:52.038438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-cf027a7d-cc06-4709-8545-416676302cc7 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:52.039603 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-cf027a7d-cc06-4709-8545-416676302cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:52.105030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 555/1125] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 04:27:52.129377 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b1876-3055-4213-9d2c-ef3249045e1b None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:52.227889 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b1876-3055-4213-9d2c-ef3249045e1b None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:52.250574 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:52.262273 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e451fa5-9b7d-4466-9f74-6d6a22cd340b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:52.262838 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e451fa5-9b7d-4466-9f74-6d6a22cd340b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:52.263121 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e451fa5-9b7d-4466-9f74-6d6a22cd340b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:52.263423 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e451fa5-9b7d-4466-9f74-6d6a22cd340b None admin] PATH_INFO: `/v3/roles/7a855ec41b944458821d76509b481268` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:52.264964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e451fa5-9b7d-4466-9f74-6d6a22cd340b None admin] RBAC: Authorizing `identity:get_role(role_id=7a855ec41b944458821d76509b481268)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:52.266365 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e451fa5-9b7d-4466-9f74-6d6a22cd340b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:52.267783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 571/1126] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:27:52 2022] GET /identity/v3/roles/7a855ec41b944458821d76509b481268 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:52.273715 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf027a7d-cc06-4709-8545-416676302cc7 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:52.323022 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:52.374050 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf027a7d-cc06-4709-8545-416676302cc7 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:52.407761 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94867be6-caae-4689-99e0-d22ea93792f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:52.408390 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94867be6-caae-4689-99e0-d22ea93792f0 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:52.408667 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94867be6-caae-4689-99e0-d22ea93792f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:52.408936 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94867be6-caae-4689-99e0-d22ea93792f0 None admin] PATH_INFO: `/v3/users/58d00046b0324af39b03b6171765b9b5` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:52.410111 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94867be6-caae-4689-99e0-d22ea93792f0 None admin] RBAC: Authorizing `identity:get_user(user_id=58d00046b0324af39b03b6171765b9b5)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:52.411525 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94867be6-caae-4689-99e0-d22ea93792f0 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:52.413103 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 556/1127] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:27:52 2022] GET /identity/v3/users/58d00046b0324af39b03b6171765b9b5 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:52.420491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e451fa5-9b7d-4466-9f74-6d6a22cd340b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:52.519511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e451fa5-9b7d-4466-9f74-6d6a22cd340b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:52.553825 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e72c52f-78d3-4737-b832-13bfbaa056d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:52.554437 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72c52f-78d3-4737-b832-13bfbaa056d1 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:52.554709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72c52f-78d3-4737-b832-13bfbaa056d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:52.554980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72c52f-78d3-4737-b832-13bfbaa056d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:52.613588 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:52.624404 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e72c52f-78d3-4737-b832-13bfbaa056d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:52.626041 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e72c52f-78d3-4737-b832-13bfbaa056d1 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:52.724109 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 572/1128] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:27:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 04:27:52.730117 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94867be6-caae-4689-99e0-d22ea93792f0 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:52.828396 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94867be6-caae-4689-99e0-d22ea93792f0 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:52.845683 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:52.862059 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b997ae-71fc-4590-b626-ccb5f48569dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:52.862686 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b997ae-71fc-4590-b626-ccb5f48569dc None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:52.862968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b997ae-71fc-4590-b626-ccb5f48569dc None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:52.863243 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b997ae-71fc-4590-b626-ccb5f48569dc None admin] PATH_INFO: `/v3/projects/7f32ef360443455fa75d77dbf975364f` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:52.864470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b997ae-71fc-4590-b626-ccb5f48569dc None admin] RBAC: Authorizing `identity:get_project(project_id=7f32ef360443455fa75d77dbf975364f)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:52.865925 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b997ae-71fc-4590-b626-ccb5f48569dc None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:52.867575 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 557/1129] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:27:52 2022] GET /identity/v3/projects/7f32ef360443455fa75d77dbf975364f => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:52.873949 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e72c52f-78d3-4737-b832-13bfbaa056d1 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:52.960760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:52.971308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e72c52f-78d3-4737-b832-13bfbaa056d1 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:53.004864 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818381a2-2d22-46f8-8818-2722b975c8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:53.005462 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818381a2-2d22-46f8-8818-2722b975c8df None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:53.005766 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818381a2-2d22-46f8-8818-2722b975c8df None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:53.006035 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818381a2-2d22-46f8-8818-2722b975c8df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:53.006811 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818381a2-2d22-46f8-8818-2722b975c8df None admin] RBAC: Adding query filter params (role.id=7a855ec41b944458821d76509b481268, scope.project.id=7f32ef360443455fa75d77dbf975364f, user.id=58d00046b0324af39b03b6171765b9b5) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:27:53.007115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818381a2-2d22-46f8-8818-2722b975c8df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:53.008526 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818381a2-2d22-46f8-8818-2722b975c8df None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:53.015614 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 573/1130] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:27:52 2022] GET /identity/v3/role_assignments?user.id=58d00046b0324af39b03b6171765b9b5&scope.project.id=7f32ef360443455fa75d77dbf975364f&role.id=7a855ec41b944458821d76509b481268 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:55.077249 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888f561-6314-4652-812a-028612983ed8 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:55.077827 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888f561-6314-4652-812a-028612983ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:55.077827 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888f561-6314-4652-812a-028612983ed8 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:55.079137 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 558/1131] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:55.084247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9d414-75a1-400f-bfeb-daa6396d90c6 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:55.084644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9d414-75a1-400f-bfeb-daa6396d90c6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:55.084792 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9d414-75a1-400f-bfeb-daa6396d90c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:55.103278 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-66b9d414-75a1-400f-bfeb-daa6396d90c6 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:55.104451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-66b9d414-75a1-400f-bfeb-daa6396d90c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:55.171921 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 574/1132] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:55.194957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888f561-6314-4652-812a-028612983ed8 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:55.300124 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888f561-6314-4652-812a-028612983ed8 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:55.334021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db734a-2052-4836-ab33-20742a14966d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:55.334608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db734a-2052-4836-ab33-20742a14966d None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:55.334884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db734a-2052-4836-ab33-20742a14966d None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:55.335151 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db734a-2052-4836-ab33-20742a14966d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:55.336283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db734a-2052-4836-ab33-20742a14966d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:55.337718 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db734a-2052-4836-ab33-20742a14966d None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:55.339270 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 559/1133] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:27:55.345523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9d414-75a1-400f-bfeb-daa6396d90c6 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:55.428008 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:55.444275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9d414-75a1-400f-bfeb-daa6396d90c6 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:55.479247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32f8a9-05f6-448e-bfc8-d1897525fe28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:55.479866 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32f8a9-05f6-448e-bfc8-d1897525fe28 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:55.480151 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32f8a9-05f6-448e-bfc8-d1897525fe28 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:55.480428 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32f8a9-05f6-448e-bfc8-d1897525fe28 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:55.481106 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32f8a9-05f6-448e-bfc8-d1897525fe28 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:55.482583 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32f8a9-05f6-448e-bfc8-d1897525fe28 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:55.508763 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 575/1134] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Aug 4 04:27:55 2022] POST /identity/v3/projects => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:27:57.569137 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a5f14-5a9b-4254-a92f-9030c54866bf None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:57.569689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a5f14-5a9b-4254-a92f-9030c54866bf None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:57.569689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a5f14-5a9b-4254-a92f-9030c54866bf None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:57.571036 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 560/1135] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:27:57.576486 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb12242-235a-4d87-ac95-f6475c80acf3 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:57.576877 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb12242-235a-4d87-ac95-f6475c80acf3 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:57.577024 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb12242-235a-4d87-ac95-f6475c80acf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:57.595124 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-0eb12242-235a-4d87-ac95-f6475c80acf3 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:27:57.596285 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0eb12242-235a-4d87-ac95-f6475c80acf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:27:57.660976 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 576/1136] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:27:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:27:57.685962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a5f14-5a9b-4254-a92f-9030c54866bf None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:57.742571 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:57.786830 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a5f14-5a9b-4254-a92f-9030c54866bf None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:57.819729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da83ec7-7552-47aa-95af-a23e436726f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:57.820345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da83ec7-7552-47aa-95af-a23e436726f7 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:57.820620 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da83ec7-7552-47aa-95af-a23e436726f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:57.820886 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da83ec7-7552-47aa-95af-a23e436726f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:57.822038 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da83ec7-7552-47aa-95af-a23e436726f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:57.823442 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da83ec7-7552-47aa-95af-a23e436726f7 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:57.824996 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 561/1137] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:27:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:27:57.830922 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb12242-235a-4d87-ac95-f6475c80acf3 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:27:57.895854 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:27:57.931115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb12242-235a-4d87-ac95-f6475c80acf3 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:27:57.964312 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c7c23-7dac-4311-b154-a10fdb84f779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:27:57.964915 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c7c23-7dac-4311-b154-a10fdb84f779 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:27:57.965194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c7c23-7dac-4311-b154-a10fdb84f779 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:27:57.965459 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c7c23-7dac-4311-b154-a10fdb84f779 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:27:57.966869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c7c23-7dac-4311-b154-a10fdb84f779 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:27:57.968388 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c7c23-7dac-4311-b154-a10fdb84f779 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:27:57.989445 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 577/1138] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Aug 4 04:27:57 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:28:00.052057 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea4de7-4ff4-447a-b07e-592ee6bb6a00 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:00.052628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea4de7-4ff4-447a-b07e-592ee6bb6a00 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:00.052628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea4de7-4ff4-447a-b07e-592ee6bb6a00 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:00.053986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 562/1139] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:28:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:28:00.061047 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3663654-5991-4b12-b35a-393417fb3c1e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:00.061474 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3663654-5991-4b12-b35a-393417fb3c1e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:00.061628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3663654-5991-4b12-b35a-393417fb3c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:00.080260 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-e3663654-5991-4b12-b35a-393417fb3c1e None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:28:00.081433 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e3663654-5991-4b12-b35a-393417fb3c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:28:00.146129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 578/1140] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:28:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:28:00.169831 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea4de7-4ff4-447a-b07e-592ee6bb6a00 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:00.269839 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea4de7-4ff4-447a-b07e-592ee6bb6a00 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:00.291611 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:00.304207 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b80f5-3acd-4150-be52-1c10c1827258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:00.304767 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b80f5-3acd-4150-be52-1c10c1827258 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:00.305045 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b80f5-3acd-4150-be52-1c10c1827258 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:00.305314 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b80f5-3acd-4150-be52-1c10c1827258 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:00.315695 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b80f5-3acd-4150-be52-1c10c1827258 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:28:00.320519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-d21b80f5-3acd-4150-be52-1c10c1827258 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:28:00.320519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:28:00.320519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:28:00.320519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:28:00.320519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:28:00.320519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:28:00.320519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:28:00.320519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:28:00.320519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:28:00.320519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:28:00.320519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:28:00.320519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:28:00.320519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:28:00.320519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:28:00.320519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:28:00.320519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:28:00.321739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:28:00.321739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:28:00.321739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:28:00.321739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:28:00.321739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:28:00.321739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:28:00.321739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:28:00.321739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:28:00.321739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:28:00.321739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:28:00.321739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:28:00.321739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:28:00.321739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:28:00.321739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:28:00.321739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:28:00.321739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:28:00.322991 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:28:00.322991 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:28:00.322991 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:28:00.322991 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:28:00.322991 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:28:00.322991 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:28:00.322991 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:28:00.322991 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:28:00.322991 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:28:00.322991 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:28:00.322991 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:28:00.322991 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:28:00.322991 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 563/1141] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:28:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:28:00.327884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3663654-5991-4b12-b35a-393417fb3c1e None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:00.378381 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:00.424643 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3663654-5991-4b12-b35a-393417fb3c1e None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:00.457410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894da37-ba9e-466c-957f-48ac4c7c0775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:00.458041 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894da37-ba9e-466c-957f-48ac4c7c0775 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:00.458322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894da37-ba9e-466c-957f-48ac4c7c0775 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:00.458593 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894da37-ba9e-466c-957f-48ac4c7c0775 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:00.459303 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b894da37-ba9e-466c-957f-48ac4c7c0775 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:28:00.459607 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b894da37-ba9e-466c-957f-48ac4c7c0775 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:00.460997 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b894da37-ba9e-466c-957f-48ac4c7c0775 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:00.468838 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 579/1142] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:28:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:28:00.474684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b80f5-3acd-4150-be52-1c10c1827258 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:00.574829 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b80f5-3acd-4150-be52-1c10c1827258 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:00.608489 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6263fc62-17ed-494f-883e-0fe2a16b7871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:00.609125 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263fc62-17ed-494f-883e-0fe2a16b7871 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:00.609404 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263fc62-17ed-494f-883e-0fe2a16b7871 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:00.609708 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263fc62-17ed-494f-883e-0fe2a16b7871 None admin] PATH_INFO: `/v3/users/09645ebac61747a1977555e062e03ab4` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:00.623440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6263fc62-17ed-494f-883e-0fe2a16b7871 None admin] RBAC: Authorizing `identity:get_user(user_id=09645ebac61747a1977555e062e03ab4)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:00.624854 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6263fc62-17ed-494f-883e-0fe2a16b7871 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:00.626543 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 564/1143] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:28:00 2022] GET /identity/v3/users/09645ebac61747a1977555e062e03ab4 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:28:00.632332 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894da37-ba9e-466c-957f-48ac4c7c0775 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:00.734527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894da37-ba9e-466c-957f-48ac4c7c0775 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:00.747367 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:00.767517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbdee8-6926-4117-b929-f13fe9f71b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:00.768123 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbdee8-6926-4117-b929-f13fe9f71b82 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:00.768402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbdee8-6926-4117-b929-f13fe9f71b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:00.768672 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbdee8-6926-4117-b929-f13fe9f71b82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:00.834224 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cbdee8-6926-4117-b929-f13fe9f71b82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:00.835579 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cbdee8-6926-4117-b929-f13fe9f71b82 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:00.930184 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 580/1144] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:28:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 04:28:00.936079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6263fc62-17ed-494f-883e-0fe2a16b7871 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:00.979132 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:01.036038 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6263fc62-17ed-494f-883e-0fe2a16b7871 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:01.068654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd7ca33-3f74-4ecd-b948-7467e1cf6822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:01.069280 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7ca33-3f74-4ecd-b948-7467e1cf6822 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:01.069593 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7ca33-3f74-4ecd-b948-7467e1cf6822 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:01.069866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7ca33-3f74-4ecd-b948-7467e1cf6822 None admin] PATH_INFO: `/v3/projects/857a9e6cc4b14f0aaeee829d2f4d38f1` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:01.071012 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd7ca33-3f74-4ecd-b948-7467e1cf6822 None admin] RBAC: Authorizing `identity:get_project(project_id=857a9e6cc4b14f0aaeee829d2f4d38f1)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:01.072415 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd7ca33-3f74-4ecd-b948-7467e1cf6822 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:01.074094 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 565/1145] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:28:00 2022] GET /identity/v3/projects/857a9e6cc4b14f0aaeee829d2f4d38f1 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:28:01.080898 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbdee8-6926-4117-b929-f13fe9f71b82 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:01.096079 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:01.177095 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbdee8-6926-4117-b929-f13fe9f71b82 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:01.209531 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76129b-52f9-4116-941b-4955db1b28e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:01.210197 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76129b-52f9-4116-941b-4955db1b28e2 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:01.210474 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76129b-52f9-4116-941b-4955db1b28e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:01.210743 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76129b-52f9-4116-941b-4955db1b28e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:01.211511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb76129b-52f9-4116-941b-4955db1b28e2 None admin] RBAC: Adding query filter params (role.id=4a6517e5890a49718303007dac0756c7, scope.project.id=857a9e6cc4b14f0aaeee829d2f4d38f1, user.id=09645ebac61747a1977555e062e03ab4) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:28:01.211816 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb76129b-52f9-4116-941b-4955db1b28e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:01.213219 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb76129b-52f9-4116-941b-4955db1b28e2 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:01.219246 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 581/1146] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:28:01 2022] GET /identity/v3/role_assignments?user.id=09645ebac61747a1977555e062e03ab4&scope.project.id=857a9e6cc4b14f0aaeee829d2f4d38f1&role.id=4a6517e5890a49718303007dac0756c7 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:28:03.181200 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e0c65-4c62-4194-84dc-8fe6dea47e0b None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:03.181785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e0c65-4c62-4194-84dc-8fe6dea47e0b None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:03.181785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e0c65-4c62-4194-84dc-8fe6dea47e0b None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:03.183104 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 566/1147] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:28:03.188917 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabecac3-f856-4901-b431-0fd36d051120 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:03.189307 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabecac3-f856-4901-b431-0fd36d051120 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:03.189458 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabecac3-f856-4901-b431-0fd36d051120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:03.207145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-fabecac3-f856-4901-b431-0fd36d051120 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:28:03.208303 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fabecac3-f856-4901-b431-0fd36d051120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:28:03.272717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 582/1148] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:28:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:28:03.295598 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e0c65-4c62-4194-84dc-8fe6dea47e0b None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:03.393986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e0c65-4c62-4194-84dc-8fe6dea47e0b None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:03.402843 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:03.428181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03167c-3bfe-4e47-9b53-460948a311e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:03.428808 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03167c-3bfe-4e47-9b53-460948a311e2 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:03.429086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03167c-3bfe-4e47-9b53-460948a311e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:03.429359 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03167c-3bfe-4e47-9b53-460948a311e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:03.439745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d03167c-3bfe-4e47-9b53-460948a311e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:28:03.444551 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-8d03167c-3bfe-4e47-9b53-460948a311e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:28:03.444551 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:28:03.444551 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:28:03.444551 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:28:03.444551 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:28:03.444551 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:28:03.444551 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:28:03.444551 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:28:03.444551 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:28:03.444551 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:28:03.444551 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:28:03.444551 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:28:03.444551 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:28:03.444551 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:28:03.444551 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:28:03.444551 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:28:03.445657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:28:03.445657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:28:03.445657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:28:03.445657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:28:03.445657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:28:03.445657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:28:03.445657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:28:03.445657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:28:03.445657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:28:03.445657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:28:03.445657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:28:03.445657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:28:03.445657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:28:03.445657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:28:03.445657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:28:03.445657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:28:03.446635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:28:03.446635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:28:03.446635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:28:03.446635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:28:03.446635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:28:03.446635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:28:03.446635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:28:03.446635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:28:03.446635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:28:03.446635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:28:03.446635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:28:03.446635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:28:03.446635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 567/1149] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:28:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:28:03.451913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabecac3-f856-4901-b431-0fd36d051120 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:03.461716 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:03.548697 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabecac3-f856-4901-b431-0fd36d051120 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:03.580454 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5be629-327f-441e-884d-301c3e6e542c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:03.581054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5be629-327f-441e-884d-301c3e6e542c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:03.581329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5be629-327f-441e-884d-301c3e6e542c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:03.581620 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5be629-327f-441e-884d-301c3e6e542c None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:03.582327 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5be629-327f-441e-884d-301c3e6e542c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:28:03.582629 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5be629-327f-441e-884d-301c3e6e542c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:03.583982 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5be629-327f-441e-884d-301c3e6e542c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:03.592331 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 583/1150] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:28:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:28:03.598154 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03167c-3bfe-4e47-9b53-460948a311e2 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:03.698610 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03167c-3bfe-4e47-9b53-460948a311e2 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:03.732408 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac443d98-e098-4ee1-b9d3-ca7d43ec1838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:03.733114 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac443d98-e098-4ee1-b9d3-ca7d43ec1838 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:03.733396 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac443d98-e098-4ee1-b9d3-ca7d43ec1838 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:03.733699 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac443d98-e098-4ee1-b9d3-ca7d43ec1838 None admin] PATH_INFO: `/v3/users/09645ebac61747a1977555e062e03ab4` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:03.735446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac443d98-e098-4ee1-b9d3-ca7d43ec1838 None admin] RBAC: Authorizing `identity:get_user(user_id=09645ebac61747a1977555e062e03ab4)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:03.736889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac443d98-e098-4ee1-b9d3-ca7d43ec1838 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:03.739764 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 568/1151] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:28:03 2022] GET /identity/v3/users/09645ebac61747a1977555e062e03ab4 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:28:03.745737 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5be629-327f-441e-884d-301c3e6e542c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:03.813614 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:03.845815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5be629-327f-441e-884d-301c3e6e542c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:03.878845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf83382-8ec4-4b45-9b16-e287749aa62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:03.879472 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf83382-8ec4-4b45-9b16-e287749aa62b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:03.879757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf83382-8ec4-4b45-9b16-e287749aa62b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:03.880029 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf83382-8ec4-4b45-9b16-e287749aa62b None admin] PATH_INFO: `/v3/projects/857a9e6cc4b14f0aaeee829d2f4d38f1` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:03.881182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf83382-8ec4-4b45-9b16-e287749aa62b None admin] RBAC: Authorizing `identity:get_project(project_id=857a9e6cc4b14f0aaeee829d2f4d38f1)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:03.882640 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf83382-8ec4-4b45-9b16-e287749aa62b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:03.884287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 584/1152] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:28:03 2022] GET /identity/v3/projects/857a9e6cc4b14f0aaeee829d2f4d38f1 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:28:03.890905 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac443d98-e098-4ee1-b9d3-ca7d43ec1838 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:03.926010 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:03.988884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac443d98-e098-4ee1-b9d3-ca7d43ec1838 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:04.022360 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0aef53-bdba-49e9-a740-2a583818914a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:04.023016 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0aef53-bdba-49e9-a740-2a583818914a None admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:04.023296 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0aef53-bdba-49e9-a740-2a583818914a None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:04.023571 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0aef53-bdba-49e9-a740-2a583818914a None admin] PATH_INFO: `/v3/projects/857a9e6cc4b14f0aaeee829d2f4d38f1/users/09645ebac61747a1977555e062e03ab4/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:04.025490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0aef53-bdba-49e9-a740-2a583818914a None admin] RBAC: Authorizing `identity:create_grant(project_id=857a9e6cc4b14f0aaeee829d2f4d38f1, user_id=09645ebac61747a1977555e062e03ab4, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:04.027009 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0aef53-bdba-49e9-a740-2a583818914a None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:04.040731 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 569/1153] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Aug 4 04:28:03 2022] PUT /identity/v3/projects/857a9e6cc4b14f0aaeee829d2f4d38f1/users/09645ebac61747a1977555e062e03ab4/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:28:06.035241 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93662b38-2936-4014-8b23-9f18cb4aa07f None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:06.035799 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93662b38-2936-4014-8b23-9f18cb4aa07f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:06.035799 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93662b38-2936-4014-8b23-9f18cb4aa07f None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:06.037128 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 585/1154] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:28:06.042316 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a799899-99ad-4c72-8350-6eb0e9b3c5ee None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:06.042734 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a799899-99ad-4c72-8350-6eb0e9b3c5ee None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:06.042859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a799899-99ad-4c72-8350-6eb0e9b3c5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:06.061940 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-5a799899-99ad-4c72-8350-6eb0e9b3c5ee None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:28:06.063104 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5a799899-99ad-4c72-8350-6eb0e9b3c5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:28:06.127272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 570/1155] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:28:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:28:06.151266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93662b38-2936-4014-8b23-9f18cb4aa07f None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:06.250345 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93662b38-2936-4014-8b23-9f18cb4aa07f None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:06.283235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649b5e4-0078-4188-bfb6-1f6060d5ac22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:06.283856 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649b5e4-0078-4188-bfb6-1f6060d5ac22 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:06.284136 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649b5e4-0078-4188-bfb6-1f6060d5ac22 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:06.284408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649b5e4-0078-4188-bfb6-1f6060d5ac22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:06.291015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:06.295706 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649b5e4-0078-4188-bfb6-1f6060d5ac22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:28:06.300846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-1649b5e4-0078-4188-bfb6-1f6060d5ac22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:28:06.300846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:28:06.300846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:28:06.300846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:28:06.300846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:28:06.300846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:28:06.300846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:28:06.300846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:28:06.300846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:28:06.300846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:28:06.300846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:28:06.300846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:28:06.300846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:28:06.300846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:28:06.300846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:28:06.300846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:28:06.301943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:28:06.301943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:28:06.301943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:28:06.301943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:28:06.301943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:28:06.301943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:28:06.301943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:28:06.301943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:28:06.301943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:28:06.301943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:28:06.301943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:28:06.301943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:28:06.301943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:28:06.301943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:28:06.301943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:28:06.301943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:28:06.303202 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:28:06.303202 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:28:06.303202 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:28:06.303202 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:28:06.303202 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:28:06.303202 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:28:06.303202 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:28:06.303202 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:28:06.303202 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:28:06.303202 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:28:06.303202 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:28:06.303202 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:28:06.303202 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 586/1156] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:28:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:28:06.308109 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a799899-99ad-4c72-8350-6eb0e9b3c5ee None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:06.334386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:06.408735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a799899-99ad-4c72-8350-6eb0e9b3c5ee None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:06.442699 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38cf277-890c-486e-b141-0b9efa8f4d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:06.443304 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cf277-890c-486e-b141-0b9efa8f4d79 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:06.443586 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cf277-890c-486e-b141-0b9efa8f4d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:06.443854 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cf277-890c-486e-b141-0b9efa8f4d79 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:06.444562 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38cf277-890c-486e-b141-0b9efa8f4d79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:28:06.444864 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38cf277-890c-486e-b141-0b9efa8f4d79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:06.446245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38cf277-890c-486e-b141-0b9efa8f4d79 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:06.454178 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 571/1157] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:28:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:28:06.460006 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649b5e4-0078-4188-bfb6-1f6060d5ac22 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:06.557780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649b5e4-0078-4188-bfb6-1f6060d5ac22 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:06.589892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9213e-cfc6-4efd-97f7-4b406b951c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:06.591137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9213e-cfc6-4efd-97f7-4b406b951c80 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:06.591527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9213e-cfc6-4efd-97f7-4b406b951c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:06.591865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9213e-cfc6-4efd-97f7-4b406b951c80 None admin] PATH_INFO: `/v3/users/09645ebac61747a1977555e062e03ab4` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:06.593032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9213e-cfc6-4efd-97f7-4b406b951c80 None admin] RBAC: Authorizing `identity:get_user(user_id=09645ebac61747a1977555e062e03ab4)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:06.594955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9213e-cfc6-4efd-97f7-4b406b951c80 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:06.596539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 587/1158] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:28:06 2022] GET /identity/v3/users/09645ebac61747a1977555e062e03ab4 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:28:06.602416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38cf277-890c-486e-b141-0b9efa8f4d79 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:06.690834 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:06.703025 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38cf277-890c-486e-b141-0b9efa8f4d79 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:06.736752 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c4c87-b93f-44cc-84f8-d0c92c86eba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:06.737363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c4c87-b93f-44cc-84f8-d0c92c86eba7 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:06.737676 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c4c87-b93f-44cc-84f8-d0c92c86eba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:06.737952 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c4c87-b93f-44cc-84f8-d0c92c86eba7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:06.802795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c4c87-b93f-44cc-84f8-d0c92c86eba7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:06.804218 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c4c87-b93f-44cc-84f8-d0c92c86eba7 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:06.886646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:06.899881 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 572/1159] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:28:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 04:28:06.906387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9213e-cfc6-4efd-97f7-4b406b951c80 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:06.970426 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:07.004847 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9213e-cfc6-4efd-97f7-4b406b951c80 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:07.038132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ef39b-dd94-4575-9465-9db522d02f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:07.038762 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ef39b-dd94-4575-9465-9db522d02f0c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:07.039042 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ef39b-dd94-4575-9465-9db522d02f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:07.039318 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ef39b-dd94-4575-9465-9db522d02f0c None admin] PATH_INFO: `/v3/projects/857a9e6cc4b14f0aaeee829d2f4d38f1` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:07.040474 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ef39b-dd94-4575-9465-9db522d02f0c None admin] RBAC: Authorizing `identity:get_project(project_id=857a9e6cc4b14f0aaeee829d2f4d38f1)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:07.042084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ef39b-dd94-4575-9465-9db522d02f0c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:07.044062 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 588/1160] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:28:06 2022] GET /identity/v3/projects/857a9e6cc4b14f0aaeee829d2f4d38f1 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:28:07.050656 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c4c87-b93f-44cc-84f8-d0c92c86eba7 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:07.146579 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c4c87-b93f-44cc-84f8-d0c92c86eba7 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:07.179713 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eee00f-944c-4f81-9b75-ddf77eb9bda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:07.180313 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eee00f-944c-4f81-9b75-ddf77eb9bda3 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:07.180595 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eee00f-944c-4f81-9b75-ddf77eb9bda3 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:07.180862 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eee00f-944c-4f81-9b75-ddf77eb9bda3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:07.181668 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eee00f-944c-4f81-9b75-ddf77eb9bda3 None admin] RBAC: Adding query filter params (role.id=4a6517e5890a49718303007dac0756c7, scope.project.id=857a9e6cc4b14f0aaeee829d2f4d38f1, user.id=09645ebac61747a1977555e062e03ab4) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:28:07.181973 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eee00f-944c-4f81-9b75-ddf77eb9bda3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:07.183368 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eee00f-944c-4f81-9b75-ddf77eb9bda3 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:07.189142 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 573/1161] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:28:07 2022] GET /identity/v3/role_assignments?user.id=09645ebac61747a1977555e062e03ab4&scope.project.id=857a9e6cc4b14f0aaeee829d2f4d38f1&role.id=4a6517e5890a49718303007dac0756c7 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:28:09.215253 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a79d50-c71c-477b-8125-3cf1154ce2fe None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:09.215963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a79d50-c71c-477b-8125-3cf1154ce2fe None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:09.215963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a79d50-c71c-477b-8125-3cf1154ce2fe None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:09.217220 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 589/1162] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:28:09.223018 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ecdd1-6d2c-47eb-9332-b9cedaf25a4a None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:09.223408 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ecdd1-6d2c-47eb-9332-b9cedaf25a4a None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:09.223576 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ecdd1-6d2c-47eb-9332-b9cedaf25a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:09.241128 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-c82ecdd1-6d2c-47eb-9332-b9cedaf25a4a None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:28:09.242346 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c82ecdd1-6d2c-47eb-9332-b9cedaf25a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:28:09.287652 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:09.308028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 574/1163] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:28:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:28:09.330538 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a79d50-c71c-477b-8125-3cf1154ce2fe None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:09.431042 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a79d50-c71c-477b-8125-3cf1154ce2fe None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:09.463742 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0198de-7e12-4b5e-8625-05ee7dfb72cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:09.464348 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0198de-7e12-4b5e-8625-05ee7dfb72cf None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:09.464631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0198de-7e12-4b5e-8625-05ee7dfb72cf None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:09.464903 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0198de-7e12-4b5e-8625-05ee7dfb72cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:09.467562 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:09.532932 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0198de-7e12-4b5e-8625-05ee7dfb72cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:09.534408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0198de-7e12-4b5e-8625-05ee7dfb72cf None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:09.631338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 590/1164] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:28:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 04:28:09.637816 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ecdd1-6d2c-47eb-9332-b9cedaf25a4a None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:09.737612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ecdd1-6d2c-47eb-9332-b9cedaf25a4a None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:09.772846 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ce008-e085-4186-adb3-de3d1e16a7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:09.773520 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ce008-e085-4186-adb3-de3d1e16a7d5 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:09.773891 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ce008-e085-4186-adb3-de3d1e16a7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:09.774193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ce008-e085-4186-adb3-de3d1e16a7d5 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:09.782828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1ce008-e085-4186-adb3-de3d1e16a7d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:09.784269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1ce008-e085-4186-adb3-de3d1e16a7d5 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:09.798698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-fe1ce008-e085-4186-adb3-de3d1e16a7d5 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 04 04:28:09.798698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:28:09.798698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 04 04:28:09.798698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 04 04:28:09.798698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:28:09.798698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:28:09.798698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:28:09.798698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:28:09.798698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 04 04:28:09.798698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:28:09.798698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:28:09.798698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:28:09.798698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:28:09.798698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:28:09.798698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:28:09.798698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:28:09.798698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:28:09.799890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:28:09.799890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:28:09.799890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:28:09.799890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:28:09.799890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:28:09.799890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:28:09.799890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:28:09.799890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:28:09.799890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:28:09.799890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:28:09.799890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:28:09.799890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:28:09.799890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:28:09.799890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:28:09.799890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:28:09.799890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:28:09.800781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:28:09.800781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:28:09.800781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:28:09.800781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:28:09.800781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:28:09.800781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:28:09.800781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:28:09.800781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:28:09.800781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:28:09.800781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:28:09.800781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 04 04:28:09.800781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:28:09.800781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 04 04:28:09.800781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:28:09.800781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 575/1165] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Aug 4 04:28:09 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:28:09.805983 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0198de-7e12-4b5e-8625-05ee7dfb72cf None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:09.839356 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:09.904170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0198de-7e12-4b5e-8625-05ee7dfb72cf None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:09.936665 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786fda50-d1fb-4d80-8e8f-f55b5a5b2e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:09.937261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fda50-d1fb-4d80-8e8f-f55b5a5b2e7c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:09.937536 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fda50-d1fb-4d80-8e8f-f55b5a5b2e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:09.937828 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fda50-d1fb-4d80-8e8f-f55b5a5b2e7c None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:09.938508 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786fda50-d1fb-4d80-8e8f-f55b5a5b2e7c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:28:09.938808 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786fda50-d1fb-4d80-8e8f-f55b5a5b2e7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:09.940253 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786fda50-d1fb-4d80-8e8f-f55b5a5b2e7c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:09.956770 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 591/1166] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Aug 4 04:28:09 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:28:11.957974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f85c45-74c4-4cfb-b40a-3171098c365c None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:11.958619 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f85c45-74c4-4cfb-b40a-3171098c365c None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:11.958619 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f85c45-74c4-4cfb-b40a-3171098c365c None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:11.959932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 576/1167] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:28:11.965955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea837da0-474c-4158-8df0-8025fe6d5f97 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:11.966456 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea837da0-474c-4158-8df0-8025fe6d5f97 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:11.966581 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea837da0-474c-4158-8df0-8025fe6d5f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:11.985026 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ea837da0-474c-4158-8df0-8025fe6d5f97 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:28:11.986230 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ea837da0-474c-4158-8df0-8025fe6d5f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:28:12.051116 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 592/1168] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:28:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:28:12.075740 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f85c45-74c4-4cfb-b40a-3171098c365c None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:12.127550 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:12.189798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f85c45-74c4-4cfb-b40a-3171098c365c None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:12.222642 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed605f-68c8-49a9-8f20-b679697ccf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:12.223180 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed605f-68c8-49a9-8f20-b679697ccf35 None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:12.223461 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed605f-68c8-49a9-8f20-b679697ccf35 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:12.223733 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed605f-68c8-49a9-8f20-b679697ccf35 None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:12.224427 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed605f-68c8-49a9-8f20-b679697ccf35 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:12.225789 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed605f-68c8-49a9-8f20-b679697ccf35 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:12.241406 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 577/1169] 10.222.0.47 () {60 vars in 1138 bytes} [Thu Aug 4 04:28:12 2022] POST /identity/v3/domains => generated 267 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:28:14.266836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cbf19-6250-4526-9ee3-75d92b47aba2 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:14.267367 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cbf19-6250-4526-9ee3-75d92b47aba2 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:14.267367 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cbf19-6250-4526-9ee3-75d92b47aba2 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:14.268744 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 593/1170] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:28:14.273975 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a5737-3bd9-491c-9a81-03b63566e763 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:14.274383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a5737-3bd9-491c-9a81-03b63566e763 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:14.274530 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a5737-3bd9-491c-9a81-03b63566e763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:14.294653 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-4c8a5737-3bd9-491c-9a81-03b63566e763 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:28:14.295828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-4c8a5737-3bd9-491c-9a81-03b63566e763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:28:14.362315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 578/1171] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:28:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:28:14.385719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310cbf19-6250-4526-9ee3-75d92b47aba2 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:14.413737 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:14.485008 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310cbf19-6250-4526-9ee3-75d92b47aba2 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:14.520531 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e7659-c360-41a8-9412-65e624e214d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:14.521102 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e7659-c360-41a8-9412-65e624e214d1 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:14.521379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e7659-c360-41a8-9412-65e624e214d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:14.521686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e7659-c360-41a8-9412-65e624e214d1 None admin] PATH_INFO: `/v3/domains/dd53cb85908542019ef06bec2403ae21` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:14.532658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e7659-c360-41a8-9412-65e624e214d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=dd53cb85908542019ef06bec2403ae21)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:14.534302 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e7659-c360-41a8-9412-65e624e214d1 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:14.535994 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 594/1172] 10.222.0.47 () {56 vars in 1183 bytes} [Thu Aug 4 04:28:14 2022] GET /identity/v3/domains/dd53cb85908542019ef06bec2403ae21 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:28:14.541788 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a5737-3bd9-491c-9a81-03b63566e763 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:14.571091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:14.639499 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a5737-3bd9-491c-9a81-03b63566e763 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:14.672243 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c1e85-5368-45e2-a724-552329b31048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:14.672846 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c1e85-5368-45e2-a724-552329b31048 None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:14.673121 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c1e85-5368-45e2-a724-552329b31048 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:14.673387 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c1e85-5368-45e2-a724-552329b31048 None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:14.674099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c1e85-5368-45e2-a724-552329b31048 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:14.675507 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c1e85-5368-45e2-a724-552329b31048 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:14.701092 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 579/1173] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Aug 4 04:28:14 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 04:28:16.724756 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d284797-d61e-4fd8-a704-e7ffec9f6e2c None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:16.725379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d284797-d61e-4fd8-a704-e7ffec9f6e2c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:16.725379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d284797-d61e-4fd8-a704-e7ffec9f6e2c None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:16.726693 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 595/1174] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:28:16.733720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e73de-35e2-4466-93ff-d207beeb5adf None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:16.734169 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e73de-35e2-4466-93ff-d207beeb5adf None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:16.734289 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e73de-35e2-4466-93ff-d207beeb5adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:16.752475 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-6c6e73de-35e2-4466-93ff-d207beeb5adf None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:28:16.753730 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-6c6e73de-35e2-4466-93ff-d207beeb5adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:28:16.818560 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 580/1175] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:28:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 04:28:16.842359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d284797-d61e-4fd8-a704-e7ffec9f6e2c None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:16.930492 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:16.943921 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d284797-d61e-4fd8-a704-e7ffec9f6e2c None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:16.976321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11770ac3-746e-454e-a893-2bc5442ee50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:16.976944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11770ac3-746e-454e-a893-2bc5442ee50f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:16.977226 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11770ac3-746e-454e-a893-2bc5442ee50f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:16.977501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11770ac3-746e-454e-a893-2bc5442ee50f None admin] PATH_INFO: `/v3/domains/dd53cb85908542019ef06bec2403ae21` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:16.978756 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11770ac3-746e-454e-a893-2bc5442ee50f None admin] RBAC: Authorizing `identity:get_domain(domain_id=dd53cb85908542019ef06bec2403ae21)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:16.980190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11770ac3-746e-454e-a893-2bc5442ee50f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:16.982322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 596/1176] 10.222.0.47 () {56 vars in 1183 bytes} [Thu Aug 4 04:28:16 2022] GET /identity/v3/domains/dd53cb85908542019ef06bec2403ae21 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:28:16.988187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e73de-35e2-4466-93ff-d207beeb5adf None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:17.003318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:17.084474 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e73de-35e2-4466-93ff-d207beeb5adf None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:17.117031 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954ebc8-21c7-4f1b-85dc-9cdb13e00594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:17.117658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954ebc8-21c7-4f1b-85dc-9cdb13e00594 None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:17.117935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954ebc8-21c7-4f1b-85dc-9cdb13e00594 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:17.118205 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954ebc8-21c7-4f1b-85dc-9cdb13e00594 None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:17.118870 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954ebc8-21c7-4f1b-85dc-9cdb13e00594 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:17.120272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954ebc8-21c7-4f1b-85dc-9cdb13e00594 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:17.141280 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 581/1177] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Aug 4 04:28:16 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:28:19.207455 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d40c0-a995-4b1a-8bc8-7bcbf38e5f17 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:19.208020 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d40c0-a995-4b1a-8bc8-7bcbf38e5f17 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:19.208020 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d40c0-a995-4b1a-8bc8-7bcbf38e5f17 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:19.209331 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 597/1178] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:28:19.214697 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ff946-f73a-40b0-9fda-eea76e8585f2 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:19.215193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ff946-f73a-40b0-9fda-eea76e8585f2 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:19.215324 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ff946-f73a-40b0-9fda-eea76e8585f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:19.234842 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-3f1ff946-f73a-40b0-9fda-eea76e8585f2 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:28:19.236002 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-3f1ff946-f73a-40b0-9fda-eea76e8585f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:28:19.296560 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:19.301585 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 582/1179] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:28:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:28:19.325056 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d40c0-a995-4b1a-8bc8-7bcbf38e5f17 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:19.423068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d40c0-a995-4b1a-8bc8-7bcbf38e5f17 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:19.456439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034ad17-f333-4ae4-84c1-e315c10056a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:19.457063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034ad17-f333-4ae4-84c1-e315c10056a3 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:19.457345 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034ad17-f333-4ae4-84c1-e315c10056a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:19.457698 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034ad17-f333-4ae4-84c1-e315c10056a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:19.467996 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8034ad17-f333-4ae4-84c1-e315c10056a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:28:19.472865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-8034ad17-f333-4ae4-84c1-e315c10056a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:28:19.472865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:28:19.472865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:28:19.472865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:28:19.472865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:28:19.472865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:28:19.472865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:28:19.472865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:28:19.472865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:28:19.472865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:28:19.472865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:28:19.472865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:28:19.472865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:28:19.472865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:28:19.472865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:28:19.472865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:28:19.474040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:28:19.474040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:28:19.474040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:28:19.474040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:28:19.474040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:28:19.474040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:28:19.474040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:28:19.474040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:28:19.474040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:28:19.474040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:28:19.474040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:28:19.474040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:28:19.474040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:28:19.474040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:28:19.474040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:28:19.474040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:28:19.475282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:28:19.475282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:28:19.475282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:28:19.475282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:28:19.475282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:28:19.475282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:28:19.475282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:28:19.475282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:28:19.475282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:28:19.475282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:28:19.475282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:28:19.475282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:28:19.475282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 598/1180] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:28:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:28:19.480133 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ff946-f73a-40b0-9fda-eea76e8585f2 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:19.577781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ff946-f73a-40b0-9fda-eea76e8585f2 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:19.610055 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51c5e8-8267-4e32-956f-bb664de0f9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:19.610666 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51c5e8-8267-4e32-956f-bb664de0f9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:19.610947 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51c5e8-8267-4e32-956f-bb664de0f9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:19.611221 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51c5e8-8267-4e32-956f-bb664de0f9b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:19.612036 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51c5e8-8267-4e32-956f-bb664de0f9b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:28:19.612341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51c5e8-8267-4e32-956f-bb664de0f9b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:19.613729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51c5e8-8267-4e32-956f-bb664de0f9b6 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:19.622248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 583/1181] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:28:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:28:19.628088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034ad17-f333-4ae4-84c1-e315c10056a3 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:19.646920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:19.727227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034ad17-f333-4ae4-84c1-e315c10056a3 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:19.760529 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f83e47-6067-4482-b834-0f3b2f199a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:19.761166 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f83e47-6067-4482-b834-0f3b2f199a21 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:19.761484 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f83e47-6067-4482-b834-0f3b2f199a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:19.761885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f83e47-6067-4482-b834-0f3b2f199a21 None admin] PATH_INFO: `/v3/users/fda190b59a7d456db2994be7347fb20c` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:19.774739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f83e47-6067-4482-b834-0f3b2f199a21 None admin] RBAC: Authorizing `identity:get_user(user_id=fda190b59a7d456db2994be7347fb20c)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:19.776154 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f83e47-6067-4482-b834-0f3b2f199a21 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:19.778006 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 599/1182] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:28:19 2022] GET /identity/v3/users/fda190b59a7d456db2994be7347fb20c => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:28:19.783796 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51c5e8-8267-4e32-956f-bb664de0f9b6 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:19.844608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:19.883851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51c5e8-8267-4e32-956f-bb664de0f9b6 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:19.916251 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbdba6b-9608-46a6-bb5c-99522e15fd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:19.916860 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdba6b-9608-46a6-bb5c-99522e15fd1d None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:19.917142 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdba6b-9608-46a6-bb5c-99522e15fd1d None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:19.917418 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdba6b-9608-46a6-bb5c-99522e15fd1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:19.983597 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbdba6b-9608-46a6-bb5c-99522e15fd1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:19.984957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbdba6b-9608-46a6-bb5c-99522e15fd1d None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:20.048500 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:20.088827 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 584/1183] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:28:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 04:28:20.095869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f83e47-6067-4482-b834-0f3b2f199a21 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:20.170992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:20.196478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f83e47-6067-4482-b834-0f3b2f199a21 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:20.230909 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccad186-8adb-4516-bffe-31ff859bc362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:20.231609 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccad186-8adb-4516-bffe-31ff859bc362 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:20.231890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccad186-8adb-4516-bffe-31ff859bc362 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:20.232161 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccad186-8adb-4516-bffe-31ff859bc362 None admin] PATH_INFO: `/v3/projects/ce069d3556674421927e6f4621a6846f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:20.237254 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccad186-8adb-4516-bffe-31ff859bc362 None admin] RBAC: Authorizing `identity:get_project(project_id=ce069d3556674421927e6f4621a6846f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:20.238649 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccad186-8adb-4516-bffe-31ff859bc362 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:20.240297 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 600/1184] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:28:20 2022] GET /identity/v3/projects/ce069d3556674421927e6f4621a6846f => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:28:20.247213 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbdba6b-9608-46a6-bb5c-99522e15fd1d None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:20.343997 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbdba6b-9608-46a6-bb5c-99522e15fd1d None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:20.377743 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee79361-64eb-455b-bb0d-0c97c6b8c9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:20.378342 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee79361-64eb-455b-bb0d-0c97c6b8c9ea None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:20.378621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee79361-64eb-455b-bb0d-0c97c6b8c9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:20.378894 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee79361-64eb-455b-bb0d-0c97c6b8c9ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:20.379673 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee79361-64eb-455b-bb0d-0c97c6b8c9ea None admin] RBAC: Adding query filter params (role.id=4a6517e5890a49718303007dac0756c7, scope.project.id=ce069d3556674421927e6f4621a6846f, user.id=fda190b59a7d456db2994be7347fb20c) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:28:20.379979 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee79361-64eb-455b-bb0d-0c97c6b8c9ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:20.381382 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee79361-64eb-455b-bb0d-0c97c6b8c9ea None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:20.387245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 585/1185] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:28:20 2022] GET /identity/v3/role_assignments?user.id=fda190b59a7d456db2994be7347fb20c&scope.project.id=ce069d3556674421927e6f4621a6846f&role.id=4a6517e5890a49718303007dac0756c7 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:28:22.380933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199ee01-8685-4ac3-99dd-74b2b167cb6e None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:22.381496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199ee01-8685-4ac3-99dd-74b2b167cb6e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:22.381496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199ee01-8685-4ac3-99dd-74b2b167cb6e None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:22.382858 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 601/1186] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:28:22.388956 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011f261-b5e8-488e-a08d-b53e73fafd40 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:22.389346 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011f261-b5e8-488e-a08d-b53e73fafd40 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:22.389501 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011f261-b5e8-488e-a08d-b53e73fafd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:22.407598 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-c011f261-b5e8-488e-a08d-b53e73fafd40 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:28:22.408766 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c011f261-b5e8-488e-a08d-b53e73fafd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:28:22.428682 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:22.476583 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 586/1187] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:28:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 04:28:22.499152 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199ee01-8685-4ac3-99dd-74b2b167cb6e None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:22.599936 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199ee01-8685-4ac3-99dd-74b2b167cb6e None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:22.633481 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89408025-02c3-499d-b699-96e1b7f6a90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:22.634077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89408025-02c3-499d-b699-96e1b7f6a90e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:22.634356 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89408025-02c3-499d-b699-96e1b7f6a90e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:22.634630 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89408025-02c3-499d-b699-96e1b7f6a90e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:22.644993 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89408025-02c3-499d-b699-96e1b7f6a90e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:28:22.649884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-89408025-02c3-499d-b699-96e1b7f6a90e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:28:22.649884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:28:22.649884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:28:22.649884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:28:22.649884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:28:22.649884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:28:22.649884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:28:22.649884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:28:22.649884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:28:22.649884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:28:22.649884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:28:22.649884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:28:22.649884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:28:22.649884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:28:22.649884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:28:22.649884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:28:22.650980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:28:22.650980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:28:22.650980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:28:22.650980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:28:22.650980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:28:22.650980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:28:22.650980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:28:22.650980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:28:22.650980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:28:22.650980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:28:22.650980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:28:22.650980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:28:22.650980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:28:22.650980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:28:22.650980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:28:22.650980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:28:22.652287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:28:22.652287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:28:22.652287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:28:22.652287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:28:22.652287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:28:22.652287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:28:22.652287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:28:22.652287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:28:22.652287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:28:22.652287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:28:22.652287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:28:22.652287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:28:22.652287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 602/1188] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:28:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:28:22.657194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011f261-b5e8-488e-a08d-b53e73fafd40 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:22.754431 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011f261-b5e8-488e-a08d-b53e73fafd40 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:22.787743 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b733612-e88a-49ee-b922-202941b43c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:22.788346 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b733612-e88a-49ee-b922-202941b43c8d None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:22.788631 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b733612-e88a-49ee-b922-202941b43c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:22.788905 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b733612-e88a-49ee-b922-202941b43c8d None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:22.789705 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b733612-e88a-49ee-b922-202941b43c8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:28:22.790010 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b733612-e88a-49ee-b922-202941b43c8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:22.791356 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b733612-e88a-49ee-b922-202941b43c8d None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:22.799762 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 587/1189] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:28:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:28:22.806013 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89408025-02c3-499d-b699-96e1b7f6a90e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:22.809617 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:22.909607 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89408025-02c3-499d-b699-96e1b7f6a90e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:22.943386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e61267-e229-4dbe-92e6-2d0534016647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:22.944039 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e61267-e229-4dbe-92e6-2d0534016647 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:22.944462 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e61267-e229-4dbe-92e6-2d0534016647 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:22.944733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e61267-e229-4dbe-92e6-2d0534016647 None admin] PATH_INFO: `/v3/users/fda190b59a7d456db2994be7347fb20c` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:22.945916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e61267-e229-4dbe-92e6-2d0534016647 None admin] RBAC: Authorizing `identity:get_user(user_id=fda190b59a7d456db2994be7347fb20c)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:22.947337 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e61267-e229-4dbe-92e6-2d0534016647 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:22.949442 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 603/1190] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:28:22 2022] GET /identity/v3/users/fda190b59a7d456db2994be7347fb20c => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:28:22.955441 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b733612-e88a-49ee-b922-202941b43c8d None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:22.974364 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:23.052590 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b733612-e88a-49ee-b922-202941b43c8d None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:23.085420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c34ad-a061-4939-85e3-e22740773fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:23.086067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c34ad-a061-4939-85e3-e22740773fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:23.086345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c34ad-a061-4939-85e3-e22740773fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:23.086618 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c34ad-a061-4939-85e3-e22740773fa9 None admin] PATH_INFO: `/v3/projects/ce069d3556674421927e6f4621a6846f` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:23.087762 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76c34ad-a061-4939-85e3-e22740773fa9 None admin] RBAC: Authorizing `identity:get_project(project_id=ce069d3556674421927e6f4621a6846f)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:23.089162 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76c34ad-a061-4939-85e3-e22740773fa9 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:23.090861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 588/1191] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:28:22 2022] GET /identity/v3/projects/ce069d3556674421927e6f4621a6846f => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:28:23.096883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e61267-e229-4dbe-92e6-2d0534016647 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:23.164595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:23.195360 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e61267-e229-4dbe-92e6-2d0534016647 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:23.228926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a67ff5-8960-456a-8335-f22c4a9d3dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:23.229608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a67ff5-8960-456a-8335-f22c4a9d3dd0 None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:23.229889 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a67ff5-8960-456a-8335-f22c4a9d3dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:23.230215 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a67ff5-8960-456a-8335-f22c4a9d3dd0 None admin] PATH_INFO: `/v3/projects/ce069d3556674421927e6f4621a6846f/users/fda190b59a7d456db2994be7347fb20c/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:23.232121 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a67ff5-8960-456a-8335-f22c4a9d3dd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce069d3556674421927e6f4621a6846f, user_id=fda190b59a7d456db2994be7347fb20c, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:23.233565 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a67ff5-8960-456a-8335-f22c4a9d3dd0 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:23.246223 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 604/1192] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Aug 4 04:28:23 2022] PUT /identity/v3/projects/ce069d3556674421927e6f4621a6846f/users/fda190b59a7d456db2994be7347fb20c/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:28:25.244026 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cdc11-f5eb-4511-99a2-104fd05e1df5 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:25.244577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cdc11-f5eb-4511-99a2-104fd05e1df5 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:25.244577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cdc11-f5eb-4511-99a2-104fd05e1df5 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:25.246540 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 589/1193] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:28:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:28:25.252076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac65fe-f3cf-4e44-941d-ad77de55ac4a None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:25.252462 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac65fe-f3cf-4e44-941d-ad77de55ac4a None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:25.252624 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac65fe-f3cf-4e44-941d-ad77de55ac4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:25.271106 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-0eac65fe-f3cf-4e44-941d-ad77de55ac4a None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:28:25.272607 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0eac65fe-f3cf-4e44-941d-ad77de55ac4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:28:25.340113 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 605/1194] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:28:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:28:25.364687 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56cdc11-f5eb-4511-99a2-104fd05e1df5 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:25.449421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:25.465657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56cdc11-f5eb-4511-99a2-104fd05e1df5 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:25.498473 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e334ed-8e82-46b7-a9f6-648cb79c7555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:25.499047 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e334ed-8e82-46b7-a9f6-648cb79c7555 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:25.499328 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e334ed-8e82-46b7-a9f6-648cb79c7555 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:25.499600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e334ed-8e82-46b7-a9f6-648cb79c7555 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:25.510188 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e334ed-8e82-46b7-a9f6-648cb79c7555 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:28:25.515197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-58e334ed-8e82-46b7-a9f6-648cb79c7555 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:28:25.515197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:28:25.515197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:28:25.515197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:28:25.515197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:28:25.515197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:28:25.515197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:28:25.515197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:28:25.515197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:28:25.515197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:28:25.515197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:28:25.515197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:28:25.515197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:28:25.515197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:28:25.515197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:28:25.515197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:28:25.516299 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:28:25.516299 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:28:25.516299 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:28:25.516299 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:28:25.516299 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:28:25.516299 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:28:25.516299 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:28:25.516299 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:28:25.516299 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:28:25.516299 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:28:25.516299 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:28:25.516299 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:28:25.516299 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:28:25.516299 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:28:25.516299 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:28:25.516299 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:28:25.517295 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:28:25.517295 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:28:25.517295 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:28:25.517295 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:28:25.517295 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:28:25.517295 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:28:25.517295 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:28:25.517295 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:28:25.517295 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:28:25.517295 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:28:25.517295 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:28:25.517295 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:28:25.517295 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 590/1195] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:28:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:28:25.522651 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac65fe-f3cf-4e44-941d-ad77de55ac4a None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:25.580106 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:25.620236 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac65fe-f3cf-4e44-941d-ad77de55ac4a None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:25.653354 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaad44d-a354-45d5-a9da-eb442a6746eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:25.653976 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaad44d-a354-45d5-a9da-eb442a6746eb None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:25.654255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaad44d-a354-45d5-a9da-eb442a6746eb None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:25.654523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaad44d-a354-45d5-a9da-eb442a6746eb None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:25.655230 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaad44d-a354-45d5-a9da-eb442a6746eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:28:25.655555 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaad44d-a354-45d5-a9da-eb442a6746eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:25.656879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaad44d-a354-45d5-a9da-eb442a6746eb None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:25.664648 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 606/1196] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:28:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:28:25.671865 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e334ed-8e82-46b7-a9f6-648cb79c7555 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:25.769127 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e334ed-8e82-46b7-a9f6-648cb79c7555 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:25.803166 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1d61f-c439-40de-b11a-a0928064c88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:25.803780 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1d61f-c439-40de-b11a-a0928064c88c None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:25.804057 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1d61f-c439-40de-b11a-a0928064c88c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:25.804325 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1d61f-c439-40de-b11a-a0928064c88c None admin] PATH_INFO: `/v3/users/fda190b59a7d456db2994be7347fb20c` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:25.805455 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c1d61f-c439-40de-b11a-a0928064c88c None admin] RBAC: Authorizing `identity:get_user(user_id=fda190b59a7d456db2994be7347fb20c)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:25.806891 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c1d61f-c439-40de-b11a-a0928064c88c None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:25.808498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 591/1197] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Aug 4 04:28:25 2022] GET /identity/v3/users/fda190b59a7d456db2994be7347fb20c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:28:25.814311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaad44d-a354-45d5-a9da-eb442a6746eb None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:25.912387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaad44d-a354-45d5-a9da-eb442a6746eb None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:25.937987 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:25.950072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad630b8f-efc0-4900-84a3-01e450a116fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:25.950688 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad630b8f-efc0-4900-84a3-01e450a116fe None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:25.950968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad630b8f-efc0-4900-84a3-01e450a116fe None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:25.951239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad630b8f-efc0-4900-84a3-01e450a116fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:26.018446 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad630b8f-efc0-4900-84a3-01e450a116fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:26.019922 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad630b8f-efc0-4900-84a3-01e450a116fe None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:26.121861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 607/1198] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:28:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 04:28:26.127822 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1d61f-c439-40de-b11a-a0928064c88c None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:26.146453 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:26.228636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1d61f-c439-40de-b11a-a0928064c88c None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:26.261288 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f629922-5a02-438c-a3b8-8edf5d3cf510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:26.262075 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f629922-5a02-438c-a3b8-8edf5d3cf510 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:26.262250 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f629922-5a02-438c-a3b8-8edf5d3cf510 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:26.262450 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f629922-5a02-438c-a3b8-8edf5d3cf510 None admin] PATH_INFO: `/v3/projects/ce069d3556674421927e6f4621a6846f` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:26.263594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f629922-5a02-438c-a3b8-8edf5d3cf510 None admin] RBAC: Authorizing `identity:get_project(project_id=ce069d3556674421927e6f4621a6846f)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:26.264994 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f629922-5a02-438c-a3b8-8edf5d3cf510 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:26.266645 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 592/1199] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Aug 4 04:28:26 2022] GET /identity/v3/projects/ce069d3556674421927e6f4621a6846f => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:28:26.273116 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad630b8f-efc0-4900-84a3-01e450a116fe None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:28:26.298531 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:26.375451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad630b8f-efc0-4900-84a3-01e450a116fe None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:28:26.409664 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff5a51-f262-4526-8857-fedbcad919be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:28:26.410262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff5a51-f262-4526-8857-fedbcad919be None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:28:26.410536 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff5a51-f262-4526-8857-fedbcad919be None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:28:26.410802 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff5a51-f262-4526-8857-fedbcad919be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:28:26.411576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff5a51-f262-4526-8857-fedbcad919be None admin] RBAC: Adding query filter params (role.id=4a6517e5890a49718303007dac0756c7, scope.project.id=ce069d3556674421927e6f4621a6846f, user.id=fda190b59a7d456db2994be7347fb20c) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:28:26.411880 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff5a51-f262-4526-8857-fedbcad919be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:28:26.413369 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff5a51-f262-4526-8857-fedbcad919be None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:28:26.419404 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 608/1200] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:28:26 2022] GET /identity/v3/role_assignments?user.id=fda190b59a7d456db2994be7347fb20c&scope.project.id=ce069d3556674421927e6f4621a6846f&role.id=4a6517e5890a49718303007dac0756c7 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:28:36.257002 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=59 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:28:36.416400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=57 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:21.869966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c22e59-dbc7-482c-87dd-abf8380aec53 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:21.869966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c22e59-dbc7-482c-87dd-abf8380aec53 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:21.869966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c22e59-dbc7-482c-87dd-abf8380aec53 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:21.869966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 593/1201] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:29:21.875468 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69531d50-e18a-4f1b-b409-860ae7bdab74 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:21.875881 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69531d50-e18a-4f1b-b409-860ae7bdab74 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:21.876028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69531d50-e18a-4f1b-b409-860ae7bdab74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:21.896138 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-69531d50-e18a-4f1b-b409-860ae7bdab74 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:29:21.897472 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-69531d50-e18a-4f1b-b409-860ae7bdab74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:29:21.986256 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 609/1202] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:29:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 04:29:22.014999 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c22e59-dbc7-482c-87dd-abf8380aec53 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:22.148576 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c22e59-dbc7-482c-87dd-abf8380aec53 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:22.189026 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f0a5c-9c05-4630-bea9-3895b062ccd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:22.189697 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f0a5c-9c05-4630-bea9-3895b062ccd5 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:22.189987 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f0a5c-9c05-4630-bea9-3895b062ccd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:22.190279 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f0a5c-9c05-4630-bea9-3895b062ccd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:22.203007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f0a5c-9c05-4630-bea9-3895b062ccd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:22.204484 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f0a5c-9c05-4630-bea9-3895b062ccd5 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:22.221097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-0b7f0a5c-9c05-4630-bea9-3895b062ccd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:22.221097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:22.221097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:22.221097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:22.221097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:22.221097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:22.221097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:22.221097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:22.221097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:22.221097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:22.221097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:22.221097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:22.221097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:29:22.221097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:29:22.221097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:29:22.222320 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:29:22.222320 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:22.222320 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:22.222320 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:22.222320 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:22.222320 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:22.222320 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:22.222320 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:22.222320 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:22.222320 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:22.222320 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:22.222320 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:22.222320 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:22.222320 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:22.222320 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:22.222320 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:22.222320 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:22.223311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:29:22.223311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:29:22.223311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:22.223311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:29:22.223311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 594/1203] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:29:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:22.229987 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69531d50-e18a-4f1b-b409-860ae7bdab74 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:22.363956 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69531d50-e18a-4f1b-b409-860ae7bdab74 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:22.409320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:22.411650 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03e2cc-3221-4a72-a405-0fb93163dc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:22.412727 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03e2cc-3221-4a72-a405-0fb93163dc17 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:22.412842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03e2cc-3221-4a72-a405-0fb93163dc17 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:22.413032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03e2cc-3221-4a72-a405-0fb93163dc17 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:22.413907 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f03e2cc-3221-4a72-a405-0fb93163dc17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:22.414373 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f03e2cc-3221-4a72-a405-0fb93163dc17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:22.416644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f03e2cc-3221-4a72-a405-0fb93163dc17 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:22.431960 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 610/1204] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:22.437798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f0a5c-9c05-4630-bea9-3895b062ccd5 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:22.509442 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:22.576855 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f0a5c-9c05-4630-bea9-3895b062ccd5 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:22.628635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ca397-ebd8-470d-b0bb-86ee449e5cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:22.629589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ca397-ebd8-470d-b0bb-86ee449e5cda None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:22.630130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ca397-ebd8-470d-b0bb-86ee449e5cda None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:22.630411 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ca397-ebd8-470d-b0bb-86ee449e5cda None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:22.631175 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ca397-ebd8-470d-b0bb-86ee449e5cda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:22.633572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ca397-ebd8-470d-b0bb-86ee449e5cda None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:22.669324 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 595/1205] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Aug 4 04:29:22 2022] POST /identity/v3/users => generated 251 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:29:24.844814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b8957-1cd4-482b-806d-c13f65230cb4 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:24.845423 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b8957-1cd4-482b-806d-c13f65230cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:24.845423 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b8957-1cd4-482b-806d-c13f65230cb4 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:24.846806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 611/1206] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:29:24.853519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7e703-ee66-4a85-97be-c37cc3415fbd None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:24.854094 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7e703-ee66-4a85-97be-c37cc3415fbd None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:24.854094 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7e703-ee66-4a85-97be-c37cc3415fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:24.873222 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-f2b7e703-ee66-4a85-97be-c37cc3415fbd None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:29:24.874510 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-f2b7e703-ee66-4a85-97be-c37cc3415fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:29:24.945665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 596/1207] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:29:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 04:29:24.969416 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b8957-1cd4-482b-806d-c13f65230cb4 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:25.105338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b8957-1cd4-482b-806d-c13f65230cb4 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:25.148748 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b94da41-896e-4fba-a667-d304a0a91ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:25.149395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94da41-896e-4fba-a667-d304a0a91ffe None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:25.149718 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94da41-896e-4fba-a667-d304a0a91ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:25.150067 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94da41-896e-4fba-a667-d304a0a91ffe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:25.161070 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b94da41-896e-4fba-a667-d304a0a91ffe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:29:25.166485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-2b94da41-896e-4fba-a667-d304a0a91ffe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:29:25.166485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:25.166485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:25.166485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:25.166485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:25.166485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:25.166485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:25.166485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:25.166485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:25.166485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:25.166485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:25.166485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:25.166485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:29:25.166485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:29:25.166485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:29:25.166485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:29:25.167540 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:29:25.167540 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:29:25.167540 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:29:25.167540 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:29:25.167540 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:25.167540 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:25.167540 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:25.167540 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:25.167540 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:25.167540 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:25.167540 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:25.167540 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:25.167540 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:25.167540 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:25.167540 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:25.167540 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:25.173266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:25.173266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:25.173266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:25.173266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:25.173266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:29:25.173266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:29:25.173266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:29:25.173266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:29:25.173266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:29:25.173266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:29:25.173266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:29:25.173266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:29:25.173266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 612/1208] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Aug 4 04:29:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:25.175876 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7e703-ee66-4a85-97be-c37cc3415fbd None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:25.220213 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:25.299078 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7e703-ee66-4a85-97be-c37cc3415fbd None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:25.342181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442db96-5a5b-478a-b6ee-765f1c888de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:25.342799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442db96-5a5b-478a-b6ee-765f1c888de9 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:25.343081 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442db96-5a5b-478a-b6ee-765f1c888de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:25.343356 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442db96-5a5b-478a-b6ee-765f1c888de9 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:25.344068 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9442db96-5a5b-478a-b6ee-765f1c888de9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:25.344375 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9442db96-5a5b-478a-b6ee-765f1c888de9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:25.345764 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9442db96-5a5b-478a-b6ee-765f1c888de9 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:25.355175 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 597/1209] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Aug 4 04:29:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:25.361401 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b94da41-896e-4fba-a667-d304a0a91ffe None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:25.424485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:25.483245 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b94da41-896e-4fba-a667-d304a0a91ffe None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:25.530276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecadd6e-9fdc-4550-8fe7-e170919900b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:25.530994 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecadd6e-9fdc-4550-8fe7-e170919900b1 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:25.531294 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecadd6e-9fdc-4550-8fe7-e170919900b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:25.531561 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecadd6e-9fdc-4550-8fe7-e170919900b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:25.545075 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecadd6e-9fdc-4550-8fe7-e170919900b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:25.546629 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecadd6e-9fdc-4550-8fe7-e170919900b1 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:25.565923 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-5ecadd6e-9fdc-4550-8fe7-e170919900b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:25.565923 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:25.565923 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:25.565923 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:25.565923 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:25.565923 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:25.565923 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:25.565923 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:25.565923 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:25.565923 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:25.565923 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:25.565923 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:25.565923 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:29:25.565923 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:29:25.565923 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:29:25.567255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:29:25.567255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:25.567255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:25.567255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:25.567255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:25.567255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:25.567255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:25.567255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:25.567255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:25.567255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:25.567255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:25.567255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:25.567255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:25.567255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:25.567255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:25.567255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:25.567255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:25.568181 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:29:25.568181 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:29:25.568181 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:25.568181 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:29:25.568181 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 613/1210] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:29:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:25.574290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442db96-5a5b-478a-b6ee-765f1c888de9 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:25.682033 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:25.686125 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442db96-5a5b-478a-b6ee-765f1c888de9 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:25.721931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f69eb-db59-47b0-8fba-5fc3da92f413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:25.722623 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f69eb-db59-47b0-8fba-5fc3da92f413 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:25.722856 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f69eb-db59-47b0-8fba-5fc3da92f413 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:25.723111 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f69eb-db59-47b0-8fba-5fc3da92f413 None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:25.723777 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f69eb-db59-47b0-8fba-5fc3da92f413 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:25.724080 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f69eb-db59-47b0-8fba-5fc3da92f413 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:25.725430 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f69eb-db59-47b0-8fba-5fc3da92f413 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:25.738574 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 598/1211] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:25.746976 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecadd6e-9fdc-4550-8fe7-e170919900b1 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:25.868954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecadd6e-9fdc-4550-8fe7-e170919900b1 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:25.870560 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:25.909882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68c875-3f4e-404a-b365-0edfcbf4d4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:25.910509 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68c875-3f4e-404a-b365-0edfcbf4d4d6 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:25.910787 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68c875-3f4e-404a-b365-0edfcbf4d4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:25.911057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68c875-3f4e-404a-b365-0edfcbf4d4d6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:25.921163 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68c875-3f4e-404a-b365-0edfcbf4d4d6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:25.921348 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68c875-3f4e-404a-b365-0edfcbf4d4d6 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:25.935954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-7e68c875-3f4e-404a-b365-0edfcbf4d4d6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 04:29:25.935954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:25.935954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:25.935954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:25.935954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:25.935954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:25.935954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:25.935954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:25.935954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:25.935954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:25.935954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:25.935954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:25.935954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:29:25.935954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:29:25.935954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:29:25.935954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:29:25.936955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:25.936955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:25.936955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:29:25.936955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:29:25.936955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:29:25.936955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:29:25.936955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:25.936955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:25.936955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:25.936955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:25.936955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:25.936955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:25.936955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:25.936955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:25.936955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:25.936955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:25.936955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:25.936955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:25.937929 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:25.937929 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:25.937929 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:29:25.937929 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:29:25.937929 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:29:25.937929 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:29:25.937929 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:29:25.937929 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:29:25.937929 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 04:29:25.937929 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:29:25.937929 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 614/1212] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:29:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:25.942977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f69eb-db59-47b0-8fba-5fc3da92f413 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:26.061189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f69eb-db59-47b0-8fba-5fc3da92f413 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:26.100423 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605890f-e084-4bc3-b9b3-9de921865952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:26.101785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605890f-e084-4bc3-b9b3-9de921865952 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:26.102134 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605890f-e084-4bc3-b9b3-9de921865952 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:26.102405 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605890f-e084-4bc3-b9b3-9de921865952 None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:26.103222 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8605890f-e084-4bc3-b9b3-9de921865952 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:26.103529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8605890f-e084-4bc3-b9b3-9de921865952 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:26.105007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8605890f-e084-4bc3-b9b3-9de921865952 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:26.123984 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 599/1213] 10.222.0.47 () {56 vars in 1143 bytes} [Thu Aug 4 04:29:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:26.130197 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68c875-3f4e-404a-b365-0edfcbf4d4d6 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:26.250587 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68c875-3f4e-404a-b365-0edfcbf4d4d6 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:26.290777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089e0c3-d131-44ca-846d-91d3c5fd8c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:26.291670 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089e0c3-d131-44ca-846d-91d3c5fd8c11 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:26.291942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089e0c3-d131-44ca-846d-91d3c5fd8c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:26.292210 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089e0c3-d131-44ca-846d-91d3c5fd8c11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:26.330820 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:26.388968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089e0c3-d131-44ca-846d-91d3c5fd8c11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:26.390842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089e0c3-d131-44ca-846d-91d3c5fd8c11 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:26.536451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 615/1214] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:29:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 04:29:26.543289 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605890f-e084-4bc3-b9b3-9de921865952 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:26.567188 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:26.671351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605890f-e084-4bc3-b9b3-9de921865952 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:26.717066 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473ebb5-7ffc-4b72-989e-e329b4df3445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:26.717809 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473ebb5-7ffc-4b72-989e-e329b4df3445 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:26.718126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473ebb5-7ffc-4b72-989e-e329b4df3445 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:26.718397 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473ebb5-7ffc-4b72-989e-e329b4df3445 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:26.733888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f473ebb5-7ffc-4b72-989e-e329b4df3445 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:26.736815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f473ebb5-7ffc-4b72-989e-e329b4df3445 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:26.753622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-f473ebb5-7ffc-4b72-989e-e329b4df3445 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:26.753622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:26.753622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:26.753622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:26.753622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:26.753622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:26.753622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:26.753622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:26.753622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:26.753622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:26.753622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:26.753622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:26.753622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:29:26.753622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:29:26.753622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:29:26.754745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:29:26.754745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:26.754745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:26.754745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:26.754745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:26.754745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:26.754745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:26.754745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:26.754745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:26.754745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:26.754745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:26.754745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:26.754745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:26.754745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:26.754745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:26.754745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:26.754745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:26.755895 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:29:26.755895 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:29:26.755895 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:26.755895 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:29:26.755895 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 600/1215] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:29:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:26.762153 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089e0c3-d131-44ca-846d-91d3c5fd8c11 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:26.839651 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:26.887144 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089e0c3-d131-44ca-846d-91d3c5fd8c11 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:26.928115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00f2e7-8564-49aa-a15d-698b98165172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:26.928854 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00f2e7-8564-49aa-a15d-698b98165172 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:26.929157 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00f2e7-8564-49aa-a15d-698b98165172 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:26.929659 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00f2e7-8564-49aa-a15d-698b98165172 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:26.930401 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a00f2e7-8564-49aa-a15d-698b98165172 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:26.930735 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a00f2e7-8564-49aa-a15d-698b98165172 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:26.932149 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a00f2e7-8564-49aa-a15d-698b98165172 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:26.949593 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 616/1216] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:29:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:26.952404 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473ebb5-7ffc-4b72-989e-e329b4df3445 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:27.031777 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:27.069032 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473ebb5-7ffc-4b72-989e-e329b4df3445 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:27.114520 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c151be26-147c-4609-8bcf-697e8e65b59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:27.115172 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c151be26-147c-4609-8bcf-697e8e65b59c None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:27.115543 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c151be26-147c-4609-8bcf-697e8e65b59c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:27.116744 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c151be26-147c-4609-8bcf-697e8e65b59c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:27.125527 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c151be26-147c-4609-8bcf-697e8e65b59c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:27.126985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c151be26-147c-4609-8bcf-697e8e65b59c None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:27.139654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-c151be26-147c-4609-8bcf-697e8e65b59c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:29:27.139654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:27.139654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:27.139654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:27.139654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:27.139654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:27.139654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:27.139654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:27.139654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:27.139654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:27.139654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:27.139654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:27.139654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:29:27.139654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:29:27.139654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:29:27.140608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:29:27.140608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:27.140608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:27.140608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:27.140608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:27.140608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:27.140608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:27.140608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:27.140608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:27.140608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:27.140608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:27.140608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:27.140608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:27.140608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:27.140608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:27.140608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:27.140608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:27.141482 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:29:27.141482 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:29:27.141482 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:29:27.141482 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:29:27.141482 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:29:27.141482 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:29:27.141482 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:29:27.141482 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:29:27.141482 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 601/1217] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:29:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:27.147058 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00f2e7-8564-49aa-a15d-698b98165172 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:27.258053 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00f2e7-8564-49aa-a15d-698b98165172 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:27.292402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:27.307217 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34303dff-ab7d-414e-bcf1-4a1e26eccc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:27.307882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34303dff-ab7d-414e-bcf1-4a1e26eccc18 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:27.308155 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34303dff-ab7d-414e-bcf1-4a1e26eccc18 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:27.308511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34303dff-ab7d-414e-bcf1-4a1e26eccc18 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:27.309246 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34303dff-ab7d-414e-bcf1-4a1e26eccc18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:27.309643 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34303dff-ab7d-414e-bcf1-4a1e26eccc18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:27.311027 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34303dff-ab7d-414e-bcf1-4a1e26eccc18 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:27.325236 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 617/1218] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:29:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:27.331768 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c151be26-147c-4609-8bcf-697e8e65b59c None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:27.448491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c151be26-147c-4609-8bcf-697e8e65b59c None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:27.483817 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:27.491115 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709239e-c564-4823-9df6-d7c6d12aa455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:27.492910 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709239e-c564-4823-9df6-d7c6d12aa455 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:27.493380 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709239e-c564-4823-9df6-d7c6d12aa455 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:27.493819 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709239e-c564-4823-9df6-d7c6d12aa455 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:27.494840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709239e-c564-4823-9df6-d7c6d12aa455 None admin] RBAC: Adding query filter params (role.id=cc86b64693b6417e90ad9361460574b0, scope.project.id=f37a5f3140ec431da8ea14568654114a, user.id=350f72f740f749f68175c1c4e8ae9d2f) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:27.495367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709239e-c564-4823-9df6-d7c6d12aa455 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:27.497074 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709239e-c564-4823-9df6-d7c6d12aa455 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:27.510377 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 602/1219] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:29:27 2022] GET /identity/v3/role_assignments?user.id=350f72f740f749f68175c1c4e8ae9d2f&scope.project.id=f37a5f3140ec431da8ea14568654114a&role.id=cc86b64693b6417e90ad9361460574b0 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:29.680148 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23553b-ad05-4d83-aad7-6798466138a1 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:29.680714 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23553b-ad05-4d83-aad7-6798466138a1 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:29.680714 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23553b-ad05-4d83-aad7-6798466138a1 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:29.682059 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 618/1220] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:29:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:29:29.687500 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19016640-8d02-42dc-80ba-4507018762c1 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:29.687998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19016640-8d02-42dc-80ba-4507018762c1 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:29.688184 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19016640-8d02-42dc-80ba-4507018762c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:29.708090 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-19016640-8d02-42dc-80ba-4507018762c1 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:29:29.709324 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-19016640-8d02-42dc-80ba-4507018762c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:29:29.788978 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 603/1221] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:29:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:29:29.813994 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23553b-ad05-4d83-aad7-6798466138a1 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:29.929477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23553b-ad05-4d83-aad7-6798466138a1 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:29.970504 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b835b512-8e44-42c2-85f8-69dc6769f0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:29.971143 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835b512-8e44-42c2-85f8-69dc6769f0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:29.971421 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835b512-8e44-42c2-85f8-69dc6769f0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:29.971708 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835b512-8e44-42c2-85f8-69dc6769f0d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:29.985538 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b835b512-8e44-42c2-85f8-69dc6769f0d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:29:29.990768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-b835b512-8e44-42c2-85f8-69dc6769f0d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:29:29.990768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:29.990768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:29.990768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:29.990768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:29.990768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:29.990768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:29.990768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:29.990768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:29.990768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:29.990768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:29.990768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:29.990768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:29:29.990768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:29:29.990768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:29:29.990768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:29:29.991683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:29:29.991683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:29:29.991683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:29:29.991683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:29:29.991683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:29.991683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:29.991683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:29.991683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:29.991683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:29.991683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:29.991683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:29.991683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:29.991683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:29.991683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:29.991683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:29.991683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:29.992512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:29.992512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:29.992512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:29.992512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:29.992512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:29:29.992512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:29:29.992512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:29:29.992512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:29:29.992512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:29:29.992512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:29:29.992512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:29:29.992512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:29:29.992512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 619/1222] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Aug 4 04:29:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:29.998433 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19016640-8d02-42dc-80ba-4507018762c1 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:30.118443 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19016640-8d02-42dc-80ba-4507018762c1 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:30.142689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:30.154355 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5187fe1f-6695-4237-b509-e19bdf444af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:30.154969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187fe1f-6695-4237-b509-e19bdf444af7 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:30.155245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187fe1f-6695-4237-b509-e19bdf444af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:30.155514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187fe1f-6695-4237-b509-e19bdf444af7 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:30.156225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5187fe1f-6695-4237-b509-e19bdf444af7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:30.156525 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5187fe1f-6695-4237-b509-e19bdf444af7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:30.157895 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5187fe1f-6695-4237-b509-e19bdf444af7 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:30.167817 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 604/1223] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Aug 4 04:29:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:30.173169 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b835b512-8e44-42c2-85f8-69dc6769f0d5 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:30.221160 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:30.293034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b835b512-8e44-42c2-85f8-69dc6769f0d5 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:30.336526 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b2670-cfc5-4df9-8937-306532aa8851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:30.337334 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b2670-cfc5-4df9-8937-306532aa8851 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:30.337838 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b2670-cfc5-4df9-8937-306532aa8851 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:30.338128 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b2670-cfc5-4df9-8937-306532aa8851 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:30.349425 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b2670-cfc5-4df9-8937-306532aa8851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:30.351905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b2670-cfc5-4df9-8937-306532aa8851 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:30.367176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-d03b2670-cfc5-4df9-8937-306532aa8851 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:30.367176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:30.367176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:30.367176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:30.367176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:30.367176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:30.367176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:30.367176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:30.367176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:30.367176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:30.367176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:30.367176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:30.367176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:29:30.367176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:29:30.367176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:29:30.368511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:29:30.368511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:30.368511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:30.368511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:30.368511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:30.368511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:30.368511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:30.368511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:30.368511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:30.368511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:30.368511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:30.368511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:30.368511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:30.368511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:30.368511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:30.368511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:30.368511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:30.369681 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:29:30.369681 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:29:30.369681 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:30.369681 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:29:30.373588 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 620/1224] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:29:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:30.374560 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5187fe1f-6695-4237-b509-e19bdf444af7 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:30.497303 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5187fe1f-6695-4237-b509-e19bdf444af7 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:30.540396 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105ec8d-de4a-422d-85f9-62ef85e9ccf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:30.541341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105ec8d-de4a-422d-85f9-62ef85e9ccf6 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:30.541671 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105ec8d-de4a-422d-85f9-62ef85e9ccf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:30.542020 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105ec8d-de4a-422d-85f9-62ef85e9ccf6 None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:30.542737 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105ec8d-de4a-422d-85f9-62ef85e9ccf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:30.543047 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105ec8d-de4a-422d-85f9-62ef85e9ccf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:30.544550 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105ec8d-de4a-422d-85f9-62ef85e9ccf6 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:30.557482 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 605/1225] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:29:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:30.563229 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b2670-cfc5-4df9-8937-306532aa8851 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:30.664392 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:30.672004 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b2670-cfc5-4df9-8937-306532aa8851 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:30.710821 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e9020-d5ed-4c89-9c6a-eb74b34fe0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:30.711534 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e9020-d5ed-4c89-9c6a-eb74b34fe0ec None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:30.711716 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e9020-d5ed-4c89-9c6a-eb74b34fe0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:30.711986 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e9020-d5ed-4c89-9c6a-eb74b34fe0ec None admin] PATH_INFO: `/v3/users/placement` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:30.720399 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e9020-d5ed-4c89-9c6a-eb74b34fe0ec None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:30.721970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e9020-d5ed-4c89-9c6a-eb74b34fe0ec None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:30.734518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-080e9020-d5ed-4c89-9c6a-eb74b34fe0ec None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 04:29:30.734518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:30.734518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:30.734518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:30.734518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:30.734518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:30.734518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:30.734518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:30.734518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:30.734518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:30.734518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:30.734518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:30.734518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:29:30.734518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:29:30.734518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:29:30.734518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:29:30.735622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:30.735622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:30.735622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:29:30.735622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:29:30.735622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:29:30.735622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:29:30.735622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:30.735622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:30.735622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:30.735622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:30.735622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:30.735622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:30.735622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:30.735622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:30.735622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:30.735622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:30.735622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:30.735622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:30.736774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:30.736774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:30.736774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:29:30.736774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:29:30.736774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:29:30.736774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:29:30.736774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:29:30.736774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:29:30.736774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 04:29:30.736774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:29:30.736774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 621/1226] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:29:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:30.742918 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105ec8d-de4a-422d-85f9-62ef85e9ccf6 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:30.793204 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:30.850484 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105ec8d-de4a-422d-85f9-62ef85e9ccf6 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:30.888418 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2bf582-932c-41dd-9f28-b6c9587a30c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:30.889034 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2bf582-932c-41dd-9f28-b6c9587a30c3 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:30.889317 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2bf582-932c-41dd-9f28-b6c9587a30c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:30.889622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2bf582-932c-41dd-9f28-b6c9587a30c3 None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:30.890417 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2bf582-932c-41dd-9f28-b6c9587a30c3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:30.890723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2bf582-932c-41dd-9f28-b6c9587a30c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:30.893367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2bf582-932c-41dd-9f28-b6c9587a30c3 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:30.914365 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 606/1227] 10.222.0.47 () {56 vars in 1143 bytes} [Thu Aug 4 04:29:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:30.922010 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e9020-d5ed-4c89-9c6a-eb74b34fe0ec None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:31.035600 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e9020-d5ed-4c89-9c6a-eb74b34fe0ec None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:31.069934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b06dc8-8358-42d8-a416-f5addc7805ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:31.070591 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b06dc8-8358-42d8-a416-f5addc7805ad None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:31.070873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b06dc8-8358-42d8-a416-f5addc7805ad None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:31.071144 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b06dc8-8358-42d8-a416-f5addc7805ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:31.086708 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b06dc8-8358-42d8-a416-f5addc7805ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:31.088765 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b06dc8-8358-42d8-a416-f5addc7805ad None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:31.108190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-b3b06dc8-8358-42d8-a416-f5addc7805ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:31.108190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:31.108190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:31.108190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:31.108190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:31.108190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:31.108190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:31.108190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:31.108190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:31.108190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:31.108190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:31.108190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:31.108190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:29:31.108190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:29:31.108190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:29:31.109262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:29:31.109262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:31.109262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:31.109262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:31.109262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:31.109262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:31.109262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:31.109262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:31.109262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:31.109262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:31.109262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:31.109262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:31.109262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:31.109262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:31.109262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:31.109262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:31.109262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:31.110215 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:29:31.110215 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:29:31.110215 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:31.110215 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:29:31.110215 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 622/1228] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:29:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:31.117292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2bf582-932c-41dd-9f28-b6c9587a30c3 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:31.220130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2bf582-932c-41dd-9f28-b6c9587a30c3 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:31.222934 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:31.261937 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f311f0-560d-4f9e-9390-20c6bf6e04e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:31.262554 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f311f0-560d-4f9e-9390-20c6bf6e04e7 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:31.262834 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f311f0-560d-4f9e-9390-20c6bf6e04e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:31.263103 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f311f0-560d-4f9e-9390-20c6bf6e04e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:31.263998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f311f0-560d-4f9e-9390-20c6bf6e04e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:31.264305 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f311f0-560d-4f9e-9390-20c6bf6e04e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:31.265703 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f311f0-560d-4f9e-9390-20c6bf6e04e7 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:31.278422 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 607/1229] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:31.284105 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b06dc8-8358-42d8-a416-f5addc7805ad None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:31.294862 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:31.398980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b06dc8-8358-42d8-a416-f5addc7805ad None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:31.437949 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119e13c-0b02-44be-b84f-1199bc59123e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:31.438599 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119e13c-0b02-44be-b84f-1199bc59123e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:31.438885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119e13c-0b02-44be-b84f-1199bc59123e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:31.439234 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119e13c-0b02-44be-b84f-1199bc59123e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:31.450751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1119e13c-0b02-44be-b84f-1199bc59123e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:31.452259 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1119e13c-0b02-44be-b84f-1199bc59123e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:31.464992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-1119e13c-0b02-44be-b84f-1199bc59123e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:29:31.464992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:31.464992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:31.464992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:31.464992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:31.464992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:31.464992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:31.464992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:31.464992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:31.464992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:31.464992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:31.464992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:31.464992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:29:31.464992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:29:31.464992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:29:31.466216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:29:31.466216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:31.466216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:31.466216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:31.466216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:31.466216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:31.466216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:31.466216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:31.466216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:31.466216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:31.466216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:31.466216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:31.466216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:31.466216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:31.466216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:31.466216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:31.466216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:31.467587 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:29:31.467587 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:29:31.467587 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:29:31.467587 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:29:31.467587 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:29:31.467587 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:29:31.467587 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:29:31.467587 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:29:31.467587 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 623/1230] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:29:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:29:31.472137 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f311f0-560d-4f9e-9390-20c6bf6e04e7 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:31.592527 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f311f0-560d-4f9e-9390-20c6bf6e04e7 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:31.633972 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e41e49-5c0c-4f70-88e4-bdab98b67e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:31.634810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e41e49-5c0c-4f70-88e4-bdab98b67e18 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:31.635120 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e41e49-5c0c-4f70-88e4-bdab98b67e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:31.635630 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e41e49-5c0c-4f70-88e4-bdab98b67e18 None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:31.636681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e41e49-5c0c-4f70-88e4-bdab98b67e18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:31.637095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e41e49-5c0c-4f70-88e4-bdab98b67e18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:31.639050 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e41e49-5c0c-4f70-88e4-bdab98b67e18 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:31.655035 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 608/1231] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:29:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:31.661137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119e13c-0b02-44be-b84f-1199bc59123e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:31.734246 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:31.773268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119e13c-0b02-44be-b84f-1199bc59123e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:31.810253 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99309d-0677-4418-aa26-ddb517c2e48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:31.810922 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99309d-0677-4418-aa26-ddb517c2e48d None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:31.811198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99309d-0677-4418-aa26-ddb517c2e48d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:31.811510 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99309d-0677-4418-aa26-ddb517c2e48d None admin] PATH_INFO: `/v3/projects/f37a5f3140ec431da8ea14568654114a/users/350f72f740f749f68175c1c4e8ae9d2f/roles/cc86b64693b6417e90ad9361460574b0` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:31.827180 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99309d-0677-4418-aa26-ddb517c2e48d None admin] RBAC: Authorizing `identity:create_grant(project_id=f37a5f3140ec431da8ea14568654114a, user_id=350f72f740f749f68175c1c4e8ae9d2f, role_id=cc86b64693b6417e90ad9361460574b0)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:31.828865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99309d-0677-4418-aa26-ddb517c2e48d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:31.842686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 624/1232] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Aug 4 04:29:31 2022] PUT /identity/v3/projects/f37a5f3140ec431da8ea14568654114a/users/350f72f740f749f68175c1c4e8ae9d2f/roles/cc86b64693b6417e90ad9361460574b0 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:29:34.031754 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e07386-376e-4c0d-9cee-7ddcf06cf40d None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:34.032312 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e07386-376e-4c0d-9cee-7ddcf06cf40d None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:34.032312 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e07386-376e-4c0d-9cee-7ddcf06cf40d None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:34.034184 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 609/1233] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:29:34.040641 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4f5ce-169d-4668-95e7-8a4264cd20d8 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:34.041345 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4f5ce-169d-4668-95e7-8a4264cd20d8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:34.041463 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4f5ce-169d-4668-95e7-8a4264cd20d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:34.061838 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-52f4f5ce-169d-4668-95e7-8a4264cd20d8 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:29:34.063211 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-52f4f5ce-169d-4668-95e7-8a4264cd20d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:29:34.144439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 625/1234] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:29:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:29:34.170970 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e07386-376e-4c0d-9cee-7ddcf06cf40d None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:34.199285 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:34.282244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e07386-376e-4c0d-9cee-7ddcf06cf40d None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:34.322582 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbf585-82ba-402c-9277-37c0682abfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:34.323227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbf585-82ba-402c-9277-37c0682abfd6 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:34.323508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbf585-82ba-402c-9277-37c0682abfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:34.323779 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbf585-82ba-402c-9277-37c0682abfd6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:34.335216 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffbf585-82ba-402c-9277-37c0682abfd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:29:34.340498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-7ffbf585-82ba-402c-9277-37c0682abfd6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:29:34.340498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:34.340498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:34.340498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:34.340498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:34.340498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:34.340498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:34.340498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:34.340498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:34.340498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:34.340498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:34.340498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:34.340498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:29:34.340498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:29:34.340498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:29:34.340498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:29:34.341646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:29:34.341646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:29:34.341646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:29:34.341646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:29:34.341646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:34.341646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:34.341646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:34.341646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:34.341646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:34.341646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:34.341646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:34.341646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:34.341646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:34.341646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:34.341646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:34.341646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:34.342661 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:34.342661 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:34.342661 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:34.342661 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:34.342661 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:29:34.342661 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:29:34.342661 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:29:34.342661 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:29:34.342661 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:29:34.342661 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:29:34.342661 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 04:29:34.342661 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:29:34.342661 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 610/1235] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Aug 4 04:29:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:34.348428 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4f5ce-169d-4668-95e7-8a4264cd20d8 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:34.404132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:34.462220 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4f5ce-169d-4668-95e7-8a4264cd20d8 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:34.500133 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b952e31-bc67-44f2-9a29-5a6ae6a44504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:34.500694 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b952e31-bc67-44f2-9a29-5a6ae6a44504 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:34.500973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b952e31-bc67-44f2-9a29-5a6ae6a44504 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:34.501276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b952e31-bc67-44f2-9a29-5a6ae6a44504 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:34.502004 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b952e31-bc67-44f2-9a29-5a6ae6a44504 None admin] RBAC: Adding query filter params (name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:34.502312 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b952e31-bc67-44f2-9a29-5a6ae6a44504 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:34.503686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b952e31-bc67-44f2-9a29-5a6ae6a44504 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:34.512892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 626/1236] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Aug 4 04:29:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:34.518681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbf585-82ba-402c-9277-37c0682abfd6 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:34.608187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:34.625658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbf585-82ba-402c-9277-37c0682abfd6 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:34.663149 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f18eff1-5148-447c-a65e-129cb9ea63fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:34.663955 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18eff1-5148-447c-a65e-129cb9ea63fb None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:34.664237 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18eff1-5148-447c-a65e-129cb9ea63fb None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:34.664573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18eff1-5148-447c-a65e-129cb9ea63fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:34.675391 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f18eff1-5148-447c-a65e-129cb9ea63fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:34.676813 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f18eff1-5148-447c-a65e-129cb9ea63fb None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:34.692211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-0f18eff1-5148-447c-a65e-129cb9ea63fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:34.692211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:34.692211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:34.692211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:34.692211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:34.692211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:34.692211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:34.692211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:34.692211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:34.692211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:34.692211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:34.692211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:34.692211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:29:34.692211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:29:34.692211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:29:34.693327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:29:34.693327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:34.693327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:34.693327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:34.693327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:34.693327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:34.693327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:34.693327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:34.693327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:34.693327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:34.693327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:34.693327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:34.693327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:34.693327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:34.693327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:34.693327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:34.693327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:34.694927 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:29:34.694927 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:29:34.694927 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:34.694927 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:29:34.694927 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 611/1237] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:29:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:34.699872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b952e31-bc67-44f2-9a29-5a6ae6a44504 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:34.807694 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b952e31-bc67-44f2-9a29-5a6ae6a44504 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:34.816508 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:34.844959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448070b-7780-4d6e-8590-c9fc42ed01ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:34.845605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448070b-7780-4d6e-8590-c9fc42ed01ba None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:34.845885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448070b-7780-4d6e-8590-c9fc42ed01ba None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:34.846150 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448070b-7780-4d6e-8590-c9fc42ed01ba None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:34.846837 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448070b-7780-4d6e-8590-c9fc42ed01ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:34.847135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448070b-7780-4d6e-8590-c9fc42ed01ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:34.848592 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448070b-7780-4d6e-8590-c9fc42ed01ba None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:34.869647 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 627/1238] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:29:34.870823 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f18eff1-5148-447c-a65e-129cb9ea63fb None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:34.983799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f18eff1-5148-447c-a65e-129cb9ea63fb None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:35.028784 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d115b-eda6-41b3-9c26-beaaea6d442f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:35.029432 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d115b-eda6-41b3-9c26-beaaea6d442f None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:35.029803 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d115b-eda6-41b3-9c26-beaaea6d442f None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:35.030080 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d115b-eda6-41b3-9c26-beaaea6d442f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:35.033485 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:35.041031 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d115b-eda6-41b3-9c26-beaaea6d442f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:35.042939 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d115b-eda6-41b3-9c26-beaaea6d442f None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:35.056804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-910d115b-eda6-41b3-9c26-beaaea6d442f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 04:29:35.056804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:35.056804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:35.056804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:35.056804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:35.056804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:35.056804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:35.056804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:35.056804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:35.056804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:35.056804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:35.056804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:35.056804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:29:35.056804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:29:35.056804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:29:35.056804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:29:35.058445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:35.058445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:35.058445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:29:35.058445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:29:35.058445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:29:35.058445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:29:35.058445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:35.058445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:35.058445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:35.058445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:35.058445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:35.058445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:35.058445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:35.058445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:35.058445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:35.058445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:35.058445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:35.058445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:35.060707 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:35.060707 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:35.060707 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:29:35.060707 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:29:35.060707 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:29:35.060707 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:29:35.060707 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:29:35.060707 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:29:35.060707 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 04:29:35.060707 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:29:35.060707 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 612/1239] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:29:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 04:29:35.065897 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448070b-7780-4d6e-8590-c9fc42ed01ba None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:35.177287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448070b-7780-4d6e-8590-c9fc42ed01ba None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:35.213208 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2b509-1eef-4280-b24a-e8f664331a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:35.214031 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2b509-1eef-4280-b24a-e8f664331a61 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:35.214337 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2b509-1eef-4280-b24a-e8f664331a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:35.214614 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2b509-1eef-4280-b24a-e8f664331a61 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:35.215541 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec2b509-1eef-4280-b24a-e8f664331a61 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:35.215848 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec2b509-1eef-4280-b24a-e8f664331a61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:35.217291 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec2b509-1eef-4280-b24a-e8f664331a61 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:35.236277 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 628/1240] 10.222.0.47 () {56 vars in 1143 bytes} [Thu Aug 4 04:29:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:35.242293 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d115b-eda6-41b3-9c26-beaaea6d442f None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:35.351633 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d115b-eda6-41b3-9c26-beaaea6d442f None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:35.392266 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81386d-70bf-45c3-a6ed-dd1219d183c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:35.392992 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81386d-70bf-45c3-a6ed-dd1219d183c1 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:35.393358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81386d-70bf-45c3-a6ed-dd1219d183c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:35.393672 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81386d-70bf-45c3-a6ed-dd1219d183c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:35.466032 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:35.472698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa81386d-70bf-45c3-a6ed-dd1219d183c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:35.474316 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa81386d-70bf-45c3-a6ed-dd1219d183c1 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:35.599097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 613/1241] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:29:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 04:29:35.601093 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2b509-1eef-4280-b24a-e8f664331a61 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:35.629311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:35.704305 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2b509-1eef-4280-b24a-e8f664331a61 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:35.738731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68248cc-b68a-4e89-b621-b4e59fb8d9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:35.739289 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68248cc-b68a-4e89-b621-b4e59fb8d9ee None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:35.739565 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68248cc-b68a-4e89-b621-b4e59fb8d9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:35.739835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68248cc-b68a-4e89-b621-b4e59fb8d9ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:35.750953 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68248cc-b68a-4e89-b621-b4e59fb8d9ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:35.752406 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68248cc-b68a-4e89-b621-b4e59fb8d9ee None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:35.766934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-a68248cc-b68a-4e89-b621-b4e59fb8d9ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:35.766934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:35.766934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:35.766934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:35.766934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:35.766934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:35.766934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:35.766934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:35.766934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:35.766934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:35.766934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:35.766934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:35.766934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:29:35.766934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:29:35.766934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:29:35.768054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:29:35.768054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:35.768054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:35.768054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:35.768054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:35.768054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:35.768054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:35.768054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:35.768054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:35.768054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:35.768054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:35.768054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:35.768054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:35.768054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:35.768054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:35.768054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:35.768054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:35.774557 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:29:35.774557 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:29:35.774557 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:35.774557 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:29:35.774557 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 629/1242] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:29:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:35.774947 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81386d-70bf-45c3-a6ed-dd1219d183c1 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:35.886563 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:35.892983 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81386d-70bf-45c3-a6ed-dd1219d183c1 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:35.933860 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833bcfb7-ce7c-46b6-8a06-b65291ac7086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:35.934626 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833bcfb7-ce7c-46b6-8a06-b65291ac7086 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:35.935029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833bcfb7-ce7c-46b6-8a06-b65291ac7086 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:35.935364 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833bcfb7-ce7c-46b6-8a06-b65291ac7086 None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:35.936190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833bcfb7-ce7c-46b6-8a06-b65291ac7086 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:35.936497 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833bcfb7-ce7c-46b6-8a06-b65291ac7086 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:35.937884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833bcfb7-ce7c-46b6-8a06-b65291ac7086 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:35.951225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 614/1243] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:35.957603 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68248cc-b68a-4e89-b621-b4e59fb8d9ee None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:36.049800 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:36.071208 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68248cc-b68a-4e89-b621-b4e59fb8d9ee None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:36.115998 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c9ede0-dfe0-47a6-a55c-f43a5e761ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:36.116785 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9ede0-dfe0-47a6-a55c-f43a5e761ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:36.117062 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9ede0-dfe0-47a6-a55c-f43a5e761ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:36.117352 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9ede0-dfe0-47a6-a55c-f43a5e761ad2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:36.125934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c9ede0-dfe0-47a6-a55c-f43a5e761ad2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:36.127684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c9ede0-dfe0-47a6-a55c-f43a5e761ad2 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:36.140963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-04c9ede0-dfe0-47a6-a55c-f43a5e761ad2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:29:36.140963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:36.140963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:36.140963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:36.140963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:36.140963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:36.140963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:36.140963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:36.140963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:36.140963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:36.140963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:36.140963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:36.140963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:29:36.140963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:29:36.140963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:29:36.142134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:29:36.142134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:36.142134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:36.142134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:36.142134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:36.142134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:36.142134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:36.142134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:36.142134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:36.142134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:36.142134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:36.142134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:36.142134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:36.142134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:36.142134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:36.142134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:36.142134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:36.143041 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:29:36.143041 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:29:36.143041 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:29:36.143041 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:29:36.143041 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:29:36.143041 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:29:36.143041 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:29:36.143041 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:29:36.143041 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 630/1244] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:29:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:36.149094 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833bcfb7-ce7c-46b6-8a06-b65291ac7086 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:36.261200 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833bcfb7-ce7c-46b6-8a06-b65291ac7086 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:36.299158 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257be814-3c05-4ad8-8a0c-e6a48596a5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:36.299775 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257be814-3c05-4ad8-8a0c-e6a48596a5ef None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:36.300099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257be814-3c05-4ad8-8a0c-e6a48596a5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:36.300315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257be814-3c05-4ad8-8a0c-e6a48596a5ef None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:36.301062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257be814-3c05-4ad8-8a0c-e6a48596a5ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:36.301371 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257be814-3c05-4ad8-8a0c-e6a48596a5ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:36.302899 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257be814-3c05-4ad8-8a0c-e6a48596a5ef None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:36.317174 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 615/1245] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:29:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:29:36.324112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c9ede0-dfe0-47a6-a55c-f43a5e761ad2 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:36.438336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c9ede0-dfe0-47a6-a55c-f43a5e761ad2 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:36.474894 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838015d8-6bed-4999-afff-e066b35c9545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:36.476070 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838015d8-6bed-4999-afff-e066b35c9545 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:36.476350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838015d8-6bed-4999-afff-e066b35c9545 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:36.476785 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838015d8-6bed-4999-afff-e066b35c9545 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:36.478089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838015d8-6bed-4999-afff-e066b35c9545 None admin] RBAC: Adding query filter params (role.id=cc86b64693b6417e90ad9361460574b0, scope.project.id=f37a5f3140ec431da8ea14568654114a, user.id=350f72f740f749f68175c1c4e8ae9d2f) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:36.478389 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838015d8-6bed-4999-afff-e066b35c9545 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:36.480083 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838015d8-6bed-4999-afff-e066b35c9545 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:36.486817 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 631/1246] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:29:36 2022] GET /identity/v3/role_assignments?user.id=350f72f740f749f68175c1c4e8ae9d2f&scope.project.id=f37a5f3140ec431da8ea14568654114a&role.id=cc86b64693b6417e90ad9361460574b0 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:29:38.697843 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344dd07-556d-4f8e-bdef-88e220dec829 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:38.698386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344dd07-556d-4f8e-bdef-88e220dec829 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:38.698472 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344dd07-556d-4f8e-bdef-88e220dec829 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:38.700007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 616/1247] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:29:38.706171 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04084e2b-03d8-4fa5-b92f-37c3135310a3 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:38.706674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04084e2b-03d8-4fa5-b92f-37c3135310a3 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:38.706798 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04084e2b-03d8-4fa5-b92f-37c3135310a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:38.714969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:38.729955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-04084e2b-03d8-4fa5-b92f-37c3135310a3 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:29:38.731316 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-04084e2b-03d8-4fa5-b92f-37c3135310a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:29:38.822907 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 632/1248] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:29:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 04:29:38.846475 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344dd07-556d-4f8e-bdef-88e220dec829 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:38.866290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:38.962192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344dd07-556d-4f8e-bdef-88e220dec829 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:39.002979 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be26f81-72c1-4040-8a91-7b6b19df129e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:39.003812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be26f81-72c1-4040-8a91-7b6b19df129e None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:39.004172 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be26f81-72c1-4040-8a91-7b6b19df129e None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:39.004450 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be26f81-72c1-4040-8a91-7b6b19df129e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:39.017657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be26f81-72c1-4040-8a91-7b6b19df129e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:29:39.023799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-3be26f81-72c1-4040-8a91-7b6b19df129e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:29:39.023799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:39.023799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:39.023799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:39.023799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:39.023799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:39.023799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:39.023799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:39.023799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:39.023799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:39.023799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:39.023799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:39.023799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:29:39.023799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:29:39.023799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:29:39.023799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:29:39.024974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:29:39.024974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:29:39.024974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:29:39.024974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:29:39.024974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:39.024974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:39.024974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:39.024974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:39.024974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:39.024974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:39.024974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:39.024974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:39.024974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:39.024974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:39.024974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:39.024974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:39.025847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:39.025847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:39.025847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:39.025847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:39.025847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:29:39.025847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:29:39.025847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:29:39.025847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:29:39.025847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:29:39.025847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:29:39.025847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:29:39.025847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:29:39.025847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 617/1249] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:29:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:39.031188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04084e2b-03d8-4fa5-b92f-37c3135310a3 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:39.139730 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04084e2b-03d8-4fa5-b92f-37c3135310a3 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:39.175979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:39.183218 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aee516-6ca3-4433-a959-cb3cd65db847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:39.184075 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aee516-6ca3-4433-a959-cb3cd65db847 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:39.184346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aee516-6ca3-4433-a959-cb3cd65db847 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:39.184680 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aee516-6ca3-4433-a959-cb3cd65db847 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:39.185588 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aee516-6ca3-4433-a959-cb3cd65db847 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:39.185905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aee516-6ca3-4433-a959-cb3cd65db847 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:39.187286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aee516-6ca3-4433-a959-cb3cd65db847 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:39.196997 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 633/1250] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:29:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:39.203768 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be26f81-72c1-4040-8a91-7b6b19df129e None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:39.288786 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:39.319409 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be26f81-72c1-4040-8a91-7b6b19df129e None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:39.359500 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdf180-d10f-4b20-b35d-245f41c2f5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:39.360220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdf180-d10f-4b20-b35d-245f41c2f5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:39.361286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdf180-d10f-4b20-b35d-245f41c2f5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:39.361672 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdf180-d10f-4b20-b35d-245f41c2f5f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:39.373781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fdf180-d10f-4b20-b35d-245f41c2f5f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:39.375756 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fdf180-d10f-4b20-b35d-245f41c2f5f9 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:39.392309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-66fdf180-d10f-4b20-b35d-245f41c2f5f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:39.392309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:39.392309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:39.392309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:39.392309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:39.392309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:39.392309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:39.392309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:39.392309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:39.392309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:39.392309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:39.392309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:39.392309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:29:39.392309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:29:39.392309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:29:39.393412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:29:39.393412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:39.393412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:39.393412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:39.393412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:39.393412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:39.393412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:39.393412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:39.393412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:39.393412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:39.393412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:39.393412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:39.393412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:39.393412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:39.393412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:39.393412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:39.393412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:39.394347 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:29:39.394347 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:29:39.394347 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:39.394347 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:29:39.394347 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 618/1251] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:29:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:39.399822 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aee516-6ca3-4433-a959-cb3cd65db847 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:39.514159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aee516-6ca3-4433-a959-cb3cd65db847 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:39.553895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d099be9-36b1-4e3a-a84d-bf1fe3a7c35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:39.554533 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d099be9-36b1-4e3a-a84d-bf1fe3a7c35f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:39.554820 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d099be9-36b1-4e3a-a84d-bf1fe3a7c35f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:39.555075 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d099be9-36b1-4e3a-a84d-bf1fe3a7c35f None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:39.555904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d099be9-36b1-4e3a-a84d-bf1fe3a7c35f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:39.556353 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d099be9-36b1-4e3a-a84d-bf1fe3a7c35f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:39.557753 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d099be9-36b1-4e3a-a84d-bf1fe3a7c35f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:39.570343 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 634/1252] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:39.577071 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdf180-d10f-4b20-b35d-245f41c2f5f9 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:39.687778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdf180-d10f-4b20-b35d-245f41c2f5f9 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:39.716301 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:39.725770 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d3b9a-6187-42c1-a097-d3455893ff53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:39.726421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d3b9a-6187-42c1-a097-d3455893ff53 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:39.726699 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d3b9a-6187-42c1-a097-d3455893ff53 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:39.726999 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d3b9a-6187-42c1-a097-d3455893ff53 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:39.735352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d3b9a-6187-42c1-a097-d3455893ff53 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:39.736777 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d3b9a-6187-42c1-a097-d3455893ff53 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:39.748677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-689d3b9a-6187-42c1-a097-d3455893ff53 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 04:29:39.748677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:39.748677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:39.748677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:39.748677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:39.748677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:39.748677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:39.748677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:39.748677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:39.748677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:39.748677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:39.748677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:39.748677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:29:39.748677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:29:39.748677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:29:39.748677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:29:39.749840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:39.749840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:39.749840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:29:39.749840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:29:39.749840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:29:39.749840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:29:39.749840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:39.749840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:39.749840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:39.749840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:39.749840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:39.749840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:39.749840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:39.749840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:39.749840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:39.749840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:39.749840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:39.749840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:39.751238 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:39.751238 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:39.751238 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:29:39.751238 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:29:39.751238 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:29:39.751238 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:29:39.751238 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:29:39.751238 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:29:39.751238 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 04:29:39.751238 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:29:39.751238 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 619/1253] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:29:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:39.756187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d099be9-36b1-4e3a-a84d-bf1fe3a7c35f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:39.821314 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:39.873230 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d099be9-36b1-4e3a-a84d-bf1fe3a7c35f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:39.914558 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da8d62-9935-4d6c-a104-87efd1fb4ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:39.915184 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da8d62-9935-4d6c-a104-87efd1fb4ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:39.915466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da8d62-9935-4d6c-a104-87efd1fb4ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:39.916029 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da8d62-9935-4d6c-a104-87efd1fb4ee5 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:39.916828 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da8d62-9935-4d6c-a104-87efd1fb4ee5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:39.917132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da8d62-9935-4d6c-a104-87efd1fb4ee5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:39.918605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da8d62-9935-4d6c-a104-87efd1fb4ee5 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:39.940875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 635/1254] 10.222.0.47 () {56 vars in 1143 bytes} [Thu Aug 4 04:29:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:39.948632 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d3b9a-6187-42c1-a097-d3455893ff53 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:40.075556 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d3b9a-6187-42c1-a097-d3455893ff53 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:40.119957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89cf81c-2fc0-4277-ade5-72e3a94e18cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:40.120574 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cf81c-2fc0-4277-ade5-72e3a94e18cc None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:40.121003 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cf81c-2fc0-4277-ade5-72e3a94e18cc None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:40.121337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cf81c-2fc0-4277-ade5-72e3a94e18cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:40.197180 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:40.214303 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89cf81c-2fc0-4277-ade5-72e3a94e18cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:40.215845 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89cf81c-2fc0-4277-ade5-72e3a94e18cc None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:40.334991 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 620/1255] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:29:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 04:29:40.336280 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da8d62-9935-4d6c-a104-87efd1fb4ee5 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:40.447166 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da8d62-9935-4d6c-a104-87efd1fb4ee5 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:40.464894 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:40.491544 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993df7d-f144-49b0-8f34-73d020ace12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:40.492277 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993df7d-f144-49b0-8f34-73d020ace12d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:40.492642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993df7d-f144-49b0-8f34-73d020ace12d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:40.492879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993df7d-f144-49b0-8f34-73d020ace12d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:40.504261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5993df7d-f144-49b0-8f34-73d020ace12d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:40.505973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5993df7d-f144-49b0-8f34-73d020ace12d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:40.524496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-5993df7d-f144-49b0-8f34-73d020ace12d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:40.524496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:40.524496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:40.524496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:40.524496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:40.524496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:40.524496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:40.524496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:40.524496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:40.524496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:40.524496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:40.524496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:40.524496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:29:40.524496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:29:40.524496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:29:40.530478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:29:40.530478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:40.530478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:40.530478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:40.530478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:40.530478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:40.530478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:40.530478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:40.530478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:40.530478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:40.530478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:40.530478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:40.530478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:40.530478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:40.530478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:40.530478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:40.530478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:40.531609 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:29:40.531609 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:29:40.531609 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:40.531609 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:29:40.531609 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 636/1256] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:29:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:40.532048 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89cf81c-2fc0-4277-ade5-72e3a94e18cc None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:40.637492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:40.652339 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89cf81c-2fc0-4277-ade5-72e3a94e18cc None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:40.692381 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081306b-26b5-4ded-825a-c9ddccbff4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:40.692997 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081306b-26b5-4ded-825a-c9ddccbff4de None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:40.693277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081306b-26b5-4ded-825a-c9ddccbff4de None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:40.694385 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081306b-26b5-4ded-825a-c9ddccbff4de None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:40.695322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081306b-26b5-4ded-825a-c9ddccbff4de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:40.695625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081306b-26b5-4ded-825a-c9ddccbff4de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:40.696989 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081306b-26b5-4ded-825a-c9ddccbff4de None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:40.708151 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 621/1257] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:29:40.715348 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993df7d-f144-49b0-8f34-73d020ace12d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:40.832611 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993df7d-f144-49b0-8f34-73d020ace12d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:40.871677 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1194e9-7352-4d29-a49f-6971b4dea39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:40.872378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1194e9-7352-4d29-a49f-6971b4dea39a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:40.872656 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1194e9-7352-4d29-a49f-6971b4dea39a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:40.872925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1194e9-7352-4d29-a49f-6971b4dea39a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:40.880316 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1194e9-7352-4d29-a49f-6971b4dea39a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:40.882100 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1194e9-7352-4d29-a49f-6971b4dea39a None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:40.893782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-fe1194e9-7352-4d29-a49f-6971b4dea39a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:29:40.893782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:40.893782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:40.893782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:40.893782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:40.893782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:40.893782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:40.893782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:40.893782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:40.893782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:40.893782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:40.893782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:40.893782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:29:40.893782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:29:40.893782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:29:40.894895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:29:40.894895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:40.894895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:40.894895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:40.894895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:40.894895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:40.894895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:40.894895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:40.894895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:40.894895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:40.894895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:40.894895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:40.894895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:40.894895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:40.894895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:40.894895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:40.894895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:40.901195 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:29:40.901195 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:29:40.901195 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:29:40.901195 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:29:40.901195 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:29:40.901195 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:29:40.901195 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:29:40.901195 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:29:40.901195 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 637/1258] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:29:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:40.901826 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081306b-26b5-4ded-825a-c9ddccbff4de None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:41.021407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081306b-26b5-4ded-825a-c9ddccbff4de None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:41.059512 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aaa50b-6403-4680-9ad2-d6db53e3290c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:41.060136 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aaa50b-6403-4680-9ad2-d6db53e3290c None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:41.060415 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aaa50b-6403-4680-9ad2-d6db53e3290c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:41.060685 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aaa50b-6403-4680-9ad2-d6db53e3290c None admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:41.061578 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aaa50b-6403-4680-9ad2-d6db53e3290c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:41.061779 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aaa50b-6403-4680-9ad2-d6db53e3290c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:41.063160 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aaa50b-6403-4680-9ad2-d6db53e3290c None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:41.075723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 622/1259] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:29:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:41.081981 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1194e9-7352-4d29-a49f-6971b4dea39a None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:41.115047 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:41.195825 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1194e9-7352-4d29-a49f-6971b4dea39a None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:41.241649 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab3c2e-aea6-48e6-acb9-629e7e9194dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:41.242263 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab3c2e-aea6-48e6-acb9-629e7e9194dc None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:41.242592 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab3c2e-aea6-48e6-acb9-629e7e9194dc None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:41.242897 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab3c2e-aea6-48e6-acb9-629e7e9194dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:41.243683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab3c2e-aea6-48e6-acb9-629e7e9194dc None admin] RBAC: Adding query filter params (role.id=4a6517e5890a49718303007dac0756c7, scope.project.id=f37a5f3140ec431da8ea14568654114a, user.id=350f72f740f749f68175c1c4e8ae9d2f) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:41.243994 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab3c2e-aea6-48e6-acb9-629e7e9194dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:41.245468 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab3c2e-aea6-48e6-acb9-629e7e9194dc None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:41.251014 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 638/1260] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:29:41 2022] GET /identity/v3/role_assignments?user.id=350f72f740f749f68175c1c4e8ae9d2f&scope.project.id=f37a5f3140ec431da8ea14568654114a&role.id=4a6517e5890a49718303007dac0756c7 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:29:43.356729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c0a5c-0ab9-49c7-b321-fc11514085e5 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:43.357295 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c0a5c-0ab9-49c7-b321-fc11514085e5 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:43.357407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c0a5c-0ab9-49c7-b321-fc11514085e5 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:43.358804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 623/1261] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:29:43.364379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc93d0b-a70f-4f5e-b933-0db5f7c57428 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:43.364810 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc93d0b-a70f-4f5e-b933-0db5f7c57428 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:43.364936 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc93d0b-a70f-4f5e-b933-0db5f7c57428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:43.383655 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-fdc93d0b-a70f-4f5e-b933-0db5f7c57428 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:29:43.384838 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fdc93d0b-a70f-4f5e-b933-0db5f7c57428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:29:43.449660 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 639/1262] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:29:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:29:43.472005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c0a5c-0ab9-49c7-b321-fc11514085e5 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:43.483200 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:43.576105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c0a5c-0ab9-49c7-b321-fc11514085e5 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:43.608186 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5aa3b-eec8-4b57-8067-84cd407101f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:43.608746 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5aa3b-eec8-4b57-8067-84cd407101f2 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:43.609022 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5aa3b-eec8-4b57-8067-84cd407101f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:43.609290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5aa3b-eec8-4b57-8067-84cd407101f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:43.619638 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5aa3b-eec8-4b57-8067-84cd407101f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:29:43.624470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-95d5aa3b-eec8-4b57-8067-84cd407101f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:29:43.624470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:43.624470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:43.624470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:43.624470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:43.624470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:43.624470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:43.624470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:43.624470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:43.624470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:43.624470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:43.624470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:43.624470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:29:43.624470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:29:43.624470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:29:43.624470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:29:43.625523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:29:43.625523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:29:43.625523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:29:43.625523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:29:43.625523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:43.625523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:43.625523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:43.625523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:43.625523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:43.625523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:43.625523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:43.625523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:43.625523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:43.625523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:43.625523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:43.625523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:43.626410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:43.626410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:43.626410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:43.626410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:43.626410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:29:43.626410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:29:43.626410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:29:43.626410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:29:43.626410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:29:43.626410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:29:43.626410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:29:43.626410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:29:43.626410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 624/1263] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:29:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:43.632261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc93d0b-a70f-4f5e-b933-0db5f7c57428 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:43.650539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:43.730457 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc93d0b-a70f-4f5e-b933-0db5f7c57428 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:43.763820 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1823915-eb80-41fa-8e57-98f22e9aaeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:43.764413 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1823915-eb80-41fa-8e57-98f22e9aaeae None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:43.764692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1823915-eb80-41fa-8e57-98f22e9aaeae None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:43.764962 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1823915-eb80-41fa-8e57-98f22e9aaeae None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:43.765692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1823915-eb80-41fa-8e57-98f22e9aaeae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:43.765994 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1823915-eb80-41fa-8e57-98f22e9aaeae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:43.767341 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1823915-eb80-41fa-8e57-98f22e9aaeae None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:43.775153 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 640/1264] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:29:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:43.780933 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5aa3b-eec8-4b57-8067-84cd407101f2 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:43.846937 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:43.880193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5aa3b-eec8-4b57-8067-84cd407101f2 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:43.913661 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f9bd3-3070-4cd6-ac60-e7ba8bf1335a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:43.914279 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f9bd3-3070-4cd6-ac60-e7ba8bf1335a None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:43.914554 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f9bd3-3070-4cd6-ac60-e7ba8bf1335a None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:43.914819 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f9bd3-3070-4cd6-ac60-e7ba8bf1335a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:43.924629 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f9bd3-3070-4cd6-ac60-e7ba8bf1335a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:43.926230 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f9bd3-3070-4cd6-ac60-e7ba8bf1335a None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:43.941438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-748f9bd3-3070-4cd6-ac60-e7ba8bf1335a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:43.941438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:43.941438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:43.941438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:43.941438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:43.941438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:43.941438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:43.941438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:43.941438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:43.941438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:43.941438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:43.941438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:43.941438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:29:43.941438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:29:43.941438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:29:43.942592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:29:43.942592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:43.942592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:43.942592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:43.942592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:43.942592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:43.942592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:43.942592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:43.942592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:43.942592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:43.942592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:43.942592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:43.942592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:43.942592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:43.942592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:43.942592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:43.942592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:43.943635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:29:43.943635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:29:43.943635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:43.943635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:29:43.943635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 625/1265] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:29:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:43.948808 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1823915-eb80-41fa-8e57-98f22e9aaeae None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:44.028677 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:44.047831 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1823915-eb80-41fa-8e57-98f22e9aaeae None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:44.081566 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec424bc7-7b7b-4ab0-b485-cf65ab1a4a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:44.082222 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec424bc7-7b7b-4ab0-b485-cf65ab1a4a40 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:44.082523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec424bc7-7b7b-4ab0-b485-cf65ab1a4a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:44.082786 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec424bc7-7b7b-4ab0-b485-cf65ab1a4a40 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:44.083524 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec424bc7-7b7b-4ab0-b485-cf65ab1a4a40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:44.083928 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec424bc7-7b7b-4ab0-b485-cf65ab1a4a40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:44.085499 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec424bc7-7b7b-4ab0-b485-cf65ab1a4a40 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:44.099611 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 641/1266] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:44.105735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f9bd3-3070-4cd6-ac60-e7ba8bf1335a None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:44.204240 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f9bd3-3070-4cd6-ac60-e7ba8bf1335a None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:44.222827 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:44.236651 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d16f8e-7465-4efa-a578-359962d68736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:44.237274 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d16f8e-7465-4efa-a578-359962d68736 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:44.237574 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d16f8e-7465-4efa-a578-359962d68736 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:44.237849 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d16f8e-7465-4efa-a578-359962d68736 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:44.246534 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d16f8e-7465-4efa-a578-359962d68736 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:44.247953 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d16f8e-7465-4efa-a578-359962d68736 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:44.258954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-27d16f8e-7465-4efa-a578-359962d68736 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 04:29:44.258954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:44.258954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:44.258954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:44.258954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:44.258954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:44.258954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:44.258954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:44.258954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:44.258954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:44.258954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:44.258954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:44.258954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:29:44.258954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:29:44.258954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:29:44.258954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:29:44.260021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:44.260021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:44.260021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:29:44.260021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:29:44.260021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:29:44.260021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:29:44.260021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:44.260021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:44.260021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:44.260021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:44.260021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:44.260021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:44.260021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:44.260021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:44.260021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:44.260021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:44.260021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:44.260021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:44.261099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:44.261099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:44.261099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:29:44.261099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:29:44.261099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:29:44.261099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:29:44.261099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:29:44.261099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:29:44.261099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 04:29:44.261099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:29:44.261099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 626/1267] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:29:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:44.265899 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec424bc7-7b7b-4ab0-b485-cf65ab1a4a40 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:44.363752 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec424bc7-7b7b-4ab0-b485-cf65ab1a4a40 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:44.396261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a9298-ca76-41b7-86da-1a4feb2be235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:44.397593 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a9298-ca76-41b7-86da-1a4feb2be235 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:44.397874 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a9298-ca76-41b7-86da-1a4feb2be235 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:44.398141 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a9298-ca76-41b7-86da-1a4feb2be235 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:44.398921 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a9298-ca76-41b7-86da-1a4feb2be235 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:44.399222 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a9298-ca76-41b7-86da-1a4feb2be235 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:44.400635 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a9298-ca76-41b7-86da-1a4feb2be235 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:44.408898 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:44.417533 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 642/1268] 10.222.0.47 () {56 vars in 1143 bytes} [Thu Aug 4 04:29:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:44.423074 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d16f8e-7465-4efa-a578-359962d68736 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:44.522840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d16f8e-7465-4efa-a578-359962d68736 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:44.556529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13b392-e81e-4a40-abbf-f144fb22c9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:44.557152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13b392-e81e-4a40-abbf-f144fb22c9b9 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:44.557428 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13b392-e81e-4a40-abbf-f144fb22c9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:44.557725 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13b392-e81e-4a40-abbf-f144fb22c9b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:44.567836 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c13b392-e81e-4a40-abbf-f144fb22c9b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:44.570258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c13b392-e81e-4a40-abbf-f144fb22c9b9 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:44.584589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-8c13b392-e81e-4a40-abbf-f144fb22c9b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:44.584589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:44.584589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:44.584589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:44.584589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:44.584589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:44.584589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:44.584589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:44.584589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:44.584589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:44.584589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:44.584589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:44.584589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:29:44.584589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:29:44.584589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:29:44.585723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:29:44.585723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:44.585723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:44.585723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:44.585723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:44.585723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:44.585723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:44.585723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:44.585723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:44.585723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:44.585723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:44.585723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:44.585723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:44.585723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:44.585723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:44.585723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:44.585723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:44.586674 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:29:44.586674 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:29:44.586674 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:44.586674 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:29:44.586674 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 627/1269] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:29:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:44.591548 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a9298-ca76-41b7-86da-1a4feb2be235 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:44.698862 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a9298-ca76-41b7-86da-1a4feb2be235 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:44.731729 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8866ce0-87fc-4773-b456-b8fa53ee93eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:44.732327 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8866ce0-87fc-4773-b456-b8fa53ee93eb None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:44.732605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8866ce0-87fc-4773-b456-b8fa53ee93eb None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:44.732874 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8866ce0-87fc-4773-b456-b8fa53ee93eb None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:44.733573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8866ce0-87fc-4773-b456-b8fa53ee93eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:44.733935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8866ce0-87fc-4773-b456-b8fa53ee93eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:44.735429 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8866ce0-87fc-4773-b456-b8fa53ee93eb None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:44.747651 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 643/1270] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:44.753263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13b392-e81e-4a40-abbf-f144fb22c9b9 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:44.785155 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:44.853499 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13b392-e81e-4a40-abbf-f144fb22c9b9 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:44.886879 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7aef0-d4db-4cb5-be59-707498fb36c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:44.887586 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7aef0-d4db-4cb5-be59-707498fb36c4 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:44.887876 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7aef0-d4db-4cb5-be59-707498fb36c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:44.888150 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7aef0-d4db-4cb5-be59-707498fb36c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:44.894622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7aef0-d4db-4cb5-be59-707498fb36c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:44.896049 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7aef0-d4db-4cb5-be59-707498fb36c4 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:44.907523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-a8d7aef0-d4db-4cb5-be59-707498fb36c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:29:44.907523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:44.907523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:44.907523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:44.907523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:44.907523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:44.907523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:44.907523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:44.907523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:44.907523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:44.907523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:44.907523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:44.907523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:29:44.907523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:29:44.907523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:29:44.908447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:29:44.908447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:44.908447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:44.908447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:44.908447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:44.908447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:44.908447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:44.908447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:44.908447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:44.908447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:44.908447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:44.908447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:44.908447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:44.908447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:44.908447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:44.908447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:44.908447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:44.909465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:29:44.909465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:29:44.909465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:29:44.909465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:29:44.909465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:29:44.909465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:29:44.909465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:29:44.909465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:29:44.909465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 628/1271] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:29:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:44.914717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8866ce0-87fc-4773-b456-b8fa53ee93eb None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:44.972112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:45.014208 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8866ce0-87fc-4773-b456-b8fa53ee93eb None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:45.048176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ee91f-1215-48ee-a00b-f6e25037acc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:45.048713 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ee91f-1215-48ee-a00b-f6e25037acc4 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:45.048994 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ee91f-1215-48ee-a00b-f6e25037acc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:45.049265 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ee91f-1215-48ee-a00b-f6e25037acc4 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:45.050039 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ee91f-1215-48ee-a00b-f6e25037acc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:45.050347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ee91f-1215-48ee-a00b-f6e25037acc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:45.051767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ee91f-1215-48ee-a00b-f6e25037acc4 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:45.063442 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 644/1272] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:29:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:45.069590 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7aef0-d4db-4cb5-be59-707498fb36c4 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:45.163009 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:45.171826 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7aef0-d4db-4cb5-be59-707498fb36c4 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:45.204412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9dd62-9740-445e-b879-abc17473f9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:45.205066 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9dd62-9740-445e-b879-abc17473f9e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:45.205343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9dd62-9740-445e-b879-abc17473f9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:45.205644 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9dd62-9740-445e-b879-abc17473f9e2 None admin] PATH_INFO: `/v3/projects/f37a5f3140ec431da8ea14568654114a/users/350f72f740f749f68175c1c4e8ae9d2f/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:45.207594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc9dd62-9740-445e-b879-abc17473f9e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=f37a5f3140ec431da8ea14568654114a, user_id=350f72f740f749f68175c1c4e8ae9d2f, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:45.209584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc9dd62-9740-445e-b879-abc17473f9e2 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:45.222028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 629/1273] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Aug 4 04:29:45 2022] PUT /identity/v3/projects/f37a5f3140ec431da8ea14568654114a/users/350f72f740f749f68175c1c4e8ae9d2f/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:29:47.246053 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad5abd-b57c-43fc-b485-492127f78a50 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:47.246581 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad5abd-b57c-43fc-b485-492127f78a50 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:47.246581 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad5abd-b57c-43fc-b485-492127f78a50 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:47.247940 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 645/1274] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:29:47.252995 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969a817-40db-454b-8d53-4540542c1627 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:47.253396 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969a817-40db-454b-8d53-4540542c1627 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:47.253669 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969a817-40db-454b-8d53-4540542c1627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:47.272593 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-a969a817-40db-454b-8d53-4540542c1627 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:29:47.273834 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-a969a817-40db-454b-8d53-4540542c1627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:29:47.339602 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 630/1275] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:29:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:29:47.363357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad5abd-b57c-43fc-b485-492127f78a50 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:47.464240 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad5abd-b57c-43fc-b485-492127f78a50 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:47.481901 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:47.498743 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb1980-9aa7-4fec-b185-11235122e621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:47.499302 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb1980-9aa7-4fec-b185-11235122e621 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:47.499579 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb1980-9aa7-4fec-b185-11235122e621 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:47.499846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb1980-9aa7-4fec-b185-11235122e621 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:47.510733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb1980-9aa7-4fec-b185-11235122e621 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 04:29:47.515590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-abdb1980-9aa7-4fec-b185-11235122e621 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:29:47.515590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:47.515590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:47.515590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:47.515590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:47.515590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:47.515590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:47.515590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:47.515590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:47.515590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:47.515590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:47.515590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:47.515590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 04:29:47.515590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 04:29:47.515590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 04:29:47.515590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 04:29:47.516683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 04:29:47.516683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 04:29:47.516683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 04:29:47.516683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 04:29:47.516683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:47.516683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:47.516683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:47.516683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:47.516683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:47.516683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:47.516683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:47.516683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:47.516683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:47.516683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:47.516683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:47.516683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:47.517674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:47.517674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:47.517674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:47.517674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:47.517674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 04:29:47.517674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 04:29:47.517674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 04:29:47.517674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 04:29:47.517674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 04:29:47.517674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 04:29:47.517674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 04:29:47.517674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:29:47.517674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 646/1276] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Aug 4 04:29:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:47.523363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a969a817-40db-454b-8d53-4540542c1627 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:47.603027 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:47.621174 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a969a817-40db-454b-8d53-4540542c1627 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:47.655336 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6720d11-ee07-4e70-a23c-78a6eab6c869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:47.655969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6720d11-ee07-4e70-a23c-78a6eab6c869 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:47.656267 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6720d11-ee07-4e70-a23c-78a6eab6c869 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:47.656552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6720d11-ee07-4e70-a23c-78a6eab6c869 None admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:47.657280 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6720d11-ee07-4e70-a23c-78a6eab6c869 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:47.657604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6720d11-ee07-4e70-a23c-78a6eab6c869 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:47.658950 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6720d11-ee07-4e70-a23c-78a6eab6c869 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:47.667107 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 631/1277] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Aug 4 04:29:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:47.672907 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb1980-9aa7-4fec-b185-11235122e621 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:47.777317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb1980-9aa7-4fec-b185-11235122e621 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:47.810174 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ad5b1-aecf-4d4a-9a72-f7eaa00611aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:47.810795 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ad5b1-aecf-4d4a-9a72-f7eaa00611aa None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:47.811071 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ad5b1-aecf-4d4a-9a72-f7eaa00611aa None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:47.811342 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ad5b1-aecf-4d4a-9a72-f7eaa00611aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:47.823115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ad5b1-aecf-4d4a-9a72-f7eaa00611aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:47.825811 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ad5b1-aecf-4d4a-9a72-f7eaa00611aa None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:47.843461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-cc0ad5b1-aecf-4d4a-9a72-f7eaa00611aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:47.843461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:47.843461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:47.843461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:47.843461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:47.843461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:47.843461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:47.843461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:47.843461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:47.843461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:47.843461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:47.843461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:47.843461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:29:47.843461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:29:47.843461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:29:47.844530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:29:47.844530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:47.844530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:47.844530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:47.844530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:47.844530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:47.844530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:47.844530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:47.844530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:47.844530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:47.844530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:47.844530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:47.844530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:47.844530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:47.844530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:47.844530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:47.844530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:47.845470 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:29:47.845470 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:29:47.845470 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:47.845470 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:29:47.845470 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 647/1278] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:29:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:47.850459 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6720d11-ee07-4e70-a23c-78a6eab6c869 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:47.959821 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6720d11-ee07-4e70-a23c-78a6eab6c869 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:47.997965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeeb599-5cef-46e2-9de7-371ac18fb2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:47.998585 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeeb599-5cef-46e2-9de7-371ac18fb2ec None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:47.998861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeeb599-5cef-46e2-9de7-371ac18fb2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:47.999131 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeeb599-5cef-46e2-9de7-371ac18fb2ec None admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:47.999810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeeb599-5cef-46e2-9de7-371ac18fb2ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:48.000107 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeeb599-5cef-46e2-9de7-371ac18fb2ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:48.001590 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeeb599-5cef-46e2-9de7-371ac18fb2ec None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:48.018478 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 632/1279] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:29:48.025210 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ad5b1-aecf-4d4a-9a72-f7eaa00611aa None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:48.057651 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:48.148493 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ad5b1-aecf-4d4a-9a72-f7eaa00611aa None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:48.182784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b64521-dda2-439b-8774-cce65058c974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:48.183421 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b64521-dda2-439b-8774-cce65058c974 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:48.183702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b64521-dda2-439b-8774-cce65058c974 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:48.183972 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b64521-dda2-439b-8774-cce65058c974 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:48.191465 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b64521-dda2-439b-8774-cce65058c974 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:48.193699 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b64521-dda2-439b-8774-cce65058c974 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:48.205001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-e4b64521-dda2-439b-8774-cce65058c974 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 04:29:48.205001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:48.205001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:48.205001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:48.205001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:48.205001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:48.205001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:48.205001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:48.205001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:48.205001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:48.205001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:48.205001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:48.205001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 04:29:48.205001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 04:29:48.205001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 04:29:48.205001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 04:29:48.206857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:48.206857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:48.206857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 04:29:48.206857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:29:48.206857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 04:29:48.206857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 04:29:48.206857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:48.206857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:48.206857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:48.206857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:48.206857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:48.206857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:48.206857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:48.206857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:48.206857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:48.206857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:48.206857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:48.206857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:48.208088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:48.208088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:48.208088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 04:29:48.208088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 04:29:48.208088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 04:29:48.208088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 04:29:48.208088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 04:29:48.208088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 04:29:48.208088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 04:29:48.208088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:29:48.208088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 648/1280] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:29:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:48.213762 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeeb599-5cef-46e2-9de7-371ac18fb2ec None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:48.217591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:48.352357 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeeb599-5cef-46e2-9de7-371ac18fb2ec None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:48.397148 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba266b-b188-4cf2-af32-69be541ac351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:48.397794 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba266b-b188-4cf2-af32-69be541ac351 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:48.398069 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba266b-b188-4cf2-af32-69be541ac351 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:48.398335 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba266b-b188-4cf2-af32-69be541ac351 None admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:48.399124 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba266b-b188-4cf2-af32-69be541ac351 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:48.399427 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba266b-b188-4cf2-af32-69be541ac351 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:48.400830 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba266b-b188-4cf2-af32-69be541ac351 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:48.422222 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 633/1281] 10.222.0.47 () {56 vars in 1143 bytes} [Thu Aug 4 04:29:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:48.428386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b64521-dda2-439b-8774-cce65058c974 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:48.523500 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:48.534012 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b64521-dda2-439b-8774-cce65058c974 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:48.569351 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a9e20-62bc-43ee-980f-fbebde08f72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:48.571607 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a9e20-62bc-43ee-980f-fbebde08f72e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:48.571892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a9e20-62bc-43ee-980f-fbebde08f72e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:48.572167 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a9e20-62bc-43ee-980f-fbebde08f72e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:48.665748 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a9e20-62bc-43ee-980f-fbebde08f72e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:48.667042 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a9e20-62bc-43ee-980f-fbebde08f72e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:48.764941 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:48.775353 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 649/1282] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Aug 4 04:29:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 04:29:48.783925 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba266b-b188-4cf2-af32-69be541ac351 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:48.845474 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:48.902046 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba266b-b188-4cf2-af32-69be541ac351 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:48.953718 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643d55b-acc9-4263-a74d-cef91092cf37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:48.954289 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643d55b-acc9-4263-a74d-cef91092cf37 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:48.954572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643d55b-acc9-4263-a74d-cef91092cf37 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:48.954855 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643d55b-acc9-4263-a74d-cef91092cf37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:48.964862 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643d55b-acc9-4263-a74d-cef91092cf37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:48.966321 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643d55b-acc9-4263-a74d-cef91092cf37 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:48.981239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-8643d55b-acc9-4263-a74d-cef91092cf37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:48.981239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:48.981239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:48.981239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:48.981239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:48.981239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:48.981239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:48.981239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:48.981239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:48.981239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:48.981239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:48.981239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:48.981239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 04:29:48.981239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 04:29:48.981239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 04:29:48.982421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 04:29:48.982421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:48.982421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:48.982421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:48.982421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:48.982421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:48.982421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:48.982421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:48.982421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:48.982421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:48.982421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:48.982421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:48.982421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:48.982421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:48.982421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:48.982421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:48.982421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:48.983584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 04:29:48.983584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 04:29:48.983584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 04:29:48.983584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:29:48.983584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 634/1283] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:29:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:48.988165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a9e20-62bc-43ee-980f-fbebde08f72e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:49.096248 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a9e20-62bc-43ee-980f-fbebde08f72e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:49.131283 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8dcc78-bb19-4bef-b681-4f6f1d3a9e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:49.131891 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8dcc78-bb19-4bef-b681-4f6f1d3a9e5b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:49.132170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8dcc78-bb19-4bef-b681-4f6f1d3a9e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:49.132439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8dcc78-bb19-4bef-b681-4f6f1d3a9e5b None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:49.133116 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8dcc78-bb19-4bef-b681-4f6f1d3a9e5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:49.133415 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8dcc78-bb19-4bef-b681-4f6f1d3a9e5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:49.134836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8dcc78-bb19-4bef-b681-4f6f1d3a9e5b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:49.145918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 650/1284] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Aug 4 04:29:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:49.151794 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643d55b-acc9-4263-a74d-cef91092cf37 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:49.249916 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643d55b-acc9-4263-a74d-cef91092cf37 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:49.263022 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:49.283615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8aeaa-8d38-4c18-8ee3-5a3de007da84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:49.284239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8aeaa-8d38-4c18-8ee3-5a3de007da84 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:49.284516 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8aeaa-8d38-4c18-8ee3-5a3de007da84 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:49.284783 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8aeaa-8d38-4c18-8ee3-5a3de007da84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:49.292217 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d8aeaa-8d38-4c18-8ee3-5a3de007da84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:49.293675 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d8aeaa-8d38-4c18-8ee3-5a3de007da84 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:49.305735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application [None req-63d8aeaa-8d38-4c18-8ee3-5a3de007da84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:29:49.305735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:49.305735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:49.305735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:49.305735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:49.305735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:49.305735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:49.305735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:49.305735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:49.305735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:49.305735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:49.305735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:49.305735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 04:29:49.305735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 04:29:49.305735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 04:29:49.306885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 04:29:49.306885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:49.306885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:49.306885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:49.306885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:49.306885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:49.306885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:49.306885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:49.306885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:49.306885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:49.306885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:49.306885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:49.306885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:49.306885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:49.306885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:49.306885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:49.306885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:49.308109 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 04:29:49.308109 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 04:29:49.308109 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 04:29:49.308109 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 04:29:49.308109 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 04:29:49.308109 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 04:29:49.308109 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 04:29:49.308109 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: ERROR keystone.server.flask.application Aug 04 04:29:49.308109 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 635/1285] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Aug 4 04:29:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:49.312757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8dcc78-bb19-4bef-b681-4f6f1d3a9e5b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:49.362289 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:49.411323 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8dcc78-bb19-4bef-b681-4f6f1d3a9e5b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:49.444728 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf01403-f654-4694-a521-86ab67e7ef94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:49.445340 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf01403-f654-4694-a521-86ab67e7ef94 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:49.445647 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf01403-f654-4694-a521-86ab67e7ef94 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:49.445924 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf01403-f654-4694-a521-86ab67e7ef94 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:49.446670 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf01403-f654-4694-a521-86ab67e7ef94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:49.446986 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf01403-f654-4694-a521-86ab67e7ef94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:49.448409 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf01403-f654-4694-a521-86ab67e7ef94 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:49.460685 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 651/1286] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Aug 4 04:29:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:49.466800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8aeaa-8d38-4c18-8ee3-5a3de007da84 None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:49.564908 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8aeaa-8d38-4c18-8ee3-5a3de007da84 None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:49.597168 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf700604-f27e-4eb1-bd49-222a9df1bf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:49.597799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf700604-f27e-4eb1-bd49-222a9df1bf3e None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:49.598080 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf700604-f27e-4eb1-bd49-222a9df1bf3e None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:49.598350 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf700604-f27e-4eb1-bd49-222a9df1bf3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:49.599179 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf700604-f27e-4eb1-bd49-222a9df1bf3e None admin] RBAC: Adding query filter params (role.id=4a6517e5890a49718303007dac0756c7, scope.project.id=f37a5f3140ec431da8ea14568654114a, user.id=350f72f740f749f68175c1c4e8ae9d2f) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:49.600138 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf700604-f27e-4eb1-bd49-222a9df1bf3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:49.601405 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf700604-f27e-4eb1-bd49-222a9df1bf3e None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:49.607112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 636/1287] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Aug 4 04:29:49 2022] GET /identity/v3/role_assignments?user.id=350f72f740f749f68175c1c4e8ae9d2f&scope.project.id=f37a5f3140ec431da8ea14568654114a&role.id=4a6517e5890a49718303007dac0756c7 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:51.654818 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c9429-b45b-4850-94a8-c86f2cf71617 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:51.655371 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c9429-b45b-4850-94a8-c86f2cf71617 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:51.655371 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c9429-b45b-4850-94a8-c86f2cf71617 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:51.656705 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 652/1288] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:29:51.661925 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161a766-714d-470d-99cc-5bf29345d22d None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:51.662399 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161a766-714d-470d-99cc-5bf29345d22d None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:51.662651 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161a766-714d-470d-99cc-5bf29345d22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:51.681309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-4161a766-714d-470d-99cc-5bf29345d22d None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:29:51.682605 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-4161a766-714d-470d-99cc-5bf29345d22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:29:51.711655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:51.748495 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 637/1289] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:29:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:29:51.770570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c9429-b45b-4850-94a8-c86f2cf71617 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:51.869190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c9429-b45b-4850-94a8-c86f2cf71617 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:51.879714 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:51.902903 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec5554-6f4f-4a65-8a8b-ee61a81befaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:51.903521 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec5554-6f4f-4a65-8a8b-ee61a81befaf None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:51.903797 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec5554-6f4f-4a65-8a8b-ee61a81befaf None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:51.904063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec5554-6f4f-4a65-8a8b-ee61a81befaf None admin] PATH_INFO: `/v3/services/placement` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:51.904683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec5554-6f4f-4a65-8a8b-ee61a81befaf None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:51.906069 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec5554-6f4f-4a65-8a8b-ee61a81befaf None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:51.915527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-0eec5554-6f4f-4a65-8a8b-ee61a81befaf None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 04:29:51.915527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:51.915527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:29:51.915527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:29:51.915527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:29:51.915527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:29:51.915527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:29:51.915527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:29:51.915527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 04:29:51.915527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:29:51.915527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:29:51.915527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:29:51.915527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:51.915527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:51.915527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:51.915527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:51.915527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:51.916751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:51.916751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:51.916751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:51.916751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:51.916751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:51.916751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:51.916751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:29:51.916751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:29:51.916751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:29:51.916751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:29:51.916751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:51.916751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:51.916751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:51.916751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:51.916751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:51.916751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:51.917840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:51.917840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:51.917840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:51.917840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:51.917840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:51.917840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:51.917840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:51.917840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:51.917840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:51.917840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:51.917840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:29:51.917840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:29:51.917840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 04:29:51.917840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:29:51.917840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 653/1290] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Aug 4 04:29:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:51.922799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161a766-714d-470d-99cc-5bf29345d22d None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:52.019185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161a766-714d-470d-99cc-5bf29345d22d None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:52.054349 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c7658-c15f-4488-8ed6-44e76fc70d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:52.054957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c7658-c15f-4488-8ed6-44e76fc70d91 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:52.055238 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c7658-c15f-4488-8ed6-44e76fc70d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:52.055511 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c7658-c15f-4488-8ed6-44e76fc70d91 None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:52.056182 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c7658-c15f-4488-8ed6-44e76fc70d91 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:52.056488 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c7658-c15f-4488-8ed6-44e76fc70d91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:52.057859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c7658-c15f-4488-8ed6-44e76fc70d91 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:52.062554 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 638/1291] 10.222.0.47 () {56 vars in 1116 bytes} [Thu Aug 4 04:29:51 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:52.068654 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec5554-6f4f-4a65-8a8b-ee61a81befaf None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:52.175772 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec5554-6f4f-4a65-8a8b-ee61a81befaf None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:52.208686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc8eb4-968b-47ec-a937-4c420a993a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:52.209283 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc8eb4-968b-47ec-a937-4c420a993a46 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:52.209582 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc8eb4-968b-47ec-a937-4c420a993a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:52.209856 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc8eb4-968b-47ec-a937-4c420a993a46 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:52.210523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc8eb4-968b-47ec-a937-4c420a993a46 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:52.210822 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc8eb4-968b-47ec-a937-4c420a993a46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:52.212173 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc8eb4-968b-47ec-a937-4c420a993a46 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:52.216421 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 654/1292] 10.222.0.47 () {56 vars in 1116 bytes} [Thu Aug 4 04:29:52 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:54.206086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928b1a0-8592-4e0f-be92-b251185516ed None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:54.206618 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928b1a0-8592-4e0f-be92-b251185516ed None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:54.206618 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928b1a0-8592-4e0f-be92-b251185516ed None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:54.208016 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 639/1293] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:29:54.213856 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec946c-30ad-4e5f-a125-442c54361a95 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:54.214261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec946c-30ad-4e5f-a125-442c54361a95 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:54.214397 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec946c-30ad-4e5f-a125-442c54361a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:54.231845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-2aec946c-30ad-4e5f-a125-442c54361a95 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:29:54.233004 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2aec946c-30ad-4e5f-a125-442c54361a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:29:54.261749 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:54.299183 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 655/1294] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:29:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:29:54.322220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928b1a0-8592-4e0f-be92-b251185516ed None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:54.353318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:54.423020 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928b1a0-8592-4e0f-be92-b251185516ed None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:54.455961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95174ee-0bf7-42ed-9538-d2fd6b7ea785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:54.456615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95174ee-0bf7-42ed-9538-d2fd6b7ea785 None admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:54.456841 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95174ee-0bf7-42ed-9538-d2fd6b7ea785 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:54.457112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95174ee-0bf7-42ed-9538-d2fd6b7ea785 None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:54.457818 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95174ee-0bf7-42ed-9538-d2fd6b7ea785 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:54.459973 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95174ee-0bf7-42ed-9538-d2fd6b7ea785 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:54.473834 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 640/1295] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Aug 4 04:29:54 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:29:56.463985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426c10a-382c-4dde-a493-b5583c439f4f None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:56.464569 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426c10a-382c-4dde-a493-b5583c439f4f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:56.464569 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426c10a-382c-4dde-a493-b5583c439f4f None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:56.465904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 656/1296] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:29:56.470991 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fdfd6-11fc-4466-8b3a-f36e1fdcbb7a None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:56.471419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fdfd6-11fc-4466-8b3a-f36e1fdcbb7a None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:56.471551 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fdfd6-11fc-4466-8b3a-f36e1fdcbb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:56.489362 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-726fdfd6-11fc-4466-8b3a-f36e1fdcbb7a None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:29:56.490655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-726fdfd6-11fc-4466-8b3a-f36e1fdcbb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:29:56.562485 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 641/1297] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:29:56 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:29:56.586040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426c10a-382c-4dde-a493-b5583c439f4f None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:56.684631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426c10a-382c-4dde-a493-b5583c439f4f None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:56.719079 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1810d2-3d44-48d2-bb24-15111e24f60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:56.719632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1810d2-3d44-48d2-bb24-15111e24f60f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:56.719908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1810d2-3d44-48d2-bb24-15111e24f60f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:56.720176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1810d2-3d44-48d2-bb24-15111e24f60f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:56.720791 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1810d2-3d44-48d2-bb24-15111e24f60f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:56.722171 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1810d2-3d44-48d2-bb24-15111e24f60f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:56.730951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-4f1810d2-3d44-48d2-bb24-15111e24f60f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 04:29:56.730951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:56.730951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:29:56.730951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:29:56.730951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:29:56.730951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:29:56.730951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:29:56.730951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:29:56.730951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 04:29:56.730951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:29:56.730951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:29:56.730951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:29:56.730951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:56.730951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:56.730951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:56.730951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:56.730951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:56.732165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:56.732165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:56.732165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:56.732165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:56.732165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:56.732165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:56.732165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:29:56.732165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:29:56.732165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:29:56.732165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:29:56.732165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:56.732165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:56.732165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:56.732165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:56.732165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:56.732165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:56.733784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:56.733784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:56.733784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:56.733784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:56.733784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:56.733784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:56.733784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:56.733784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:56.733784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:56.733784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:56.733784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:29:56.733784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:29:56.733784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 04:29:56.733784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:29:56.733784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 657/1298] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Aug 4 04:29:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:56.741522 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726fdfd6-11fc-4466-8b3a-f36e1fdcbb7a None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:56.760306 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:56.840063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726fdfd6-11fc-4466-8b3a-f36e1fdcbb7a None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:56.873311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86cf8e-e134-4165-ac00-3c4eeb9f8c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:56.873940 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86cf8e-e134-4165-ac00-3c4eeb9f8c1c None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:56.874216 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86cf8e-e134-4165-ac00-3c4eeb9f8c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:56.874482 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86cf8e-e134-4165-ac00-3c4eeb9f8c1c None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:56.875143 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86cf8e-e134-4165-ac00-3c4eeb9f8c1c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:56.875441 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86cf8e-e134-4165-ac00-3c4eeb9f8c1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:56.876947 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86cf8e-e134-4165-ac00-3c4eeb9f8c1c None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:56.882674 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 642/1299] 10.222.0.47 () {56 vars in 1116 bytes} [Thu Aug 4 04:29:56 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:56.888637 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1810d2-3d44-48d2-bb24-15111e24f60f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:56.930008 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:56.998886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1810d2-3d44-48d2-bb24-15111e24f60f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:57.031996 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31fd8fc-53c6-431b-b2e9-157a1776c0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:57.032599 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fd8fc-53c6-431b-b2e9-157a1776c0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:57.032878 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fd8fc-53c6-431b-b2e9-157a1776c0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:57.033148 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fd8fc-53c6-431b-b2e9-157a1776c0f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:57.033932 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31fd8fc-53c6-431b-b2e9-157a1776c0f2 None admin] RBAC: Adding query filter params (interface=public, service_id=7721b32e4b1344d3ad7a6595aae400fc, region_id=RegionOne) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:57.034242 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31fd8fc-53c6-431b-b2e9-157a1776c0f2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:57.035594 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31fd8fc-53c6-431b-b2e9-157a1776c0f2 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:57.041189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 658/1300] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Aug 4 04:29:56 2022] GET /identity/v3/endpoints?service_id=7721b32e4b1344d3ad7a6595aae400fc&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:29:57.046824 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86cf8e-e134-4165-ac00-3c4eeb9f8c1c None admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:57.141282 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:57.162760 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86cf8e-e134-4165-ac00-3c4eeb9f8c1c None admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:57.197367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9377d-4a79-4dd3-8452-4c1875f29b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:57.197995 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9377d-4a79-4dd3-8452-4c1875f29b39 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:57.198271 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9377d-4a79-4dd3-8452-4c1875f29b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:57.198537 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9377d-4a79-4dd3-8452-4c1875f29b39 None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:57.199176 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9377d-4a79-4dd3-8452-4c1875f29b39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:57.200513 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9377d-4a79-4dd3-8452-4c1875f29b39 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:57.207132 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 643/1301] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Aug 4 04:29:57 2022] GET /identity/v3/services => generated 2191 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:29:59.177444 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c402f6-c4d2-4087-9d9a-27e273a35fee None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:59.178024 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c402f6-c4d2-4087-9d9a-27e273a35fee None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:59.178197 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c402f6-c4d2-4087-9d9a-27e273a35fee None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:59.179549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 659/1302] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:29:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:29:59.184562 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d2dc9-6556-49db-b8fc-ce81c27c67f7 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:59.184958 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d2dc9-6556-49db-b8fc-ce81c27c67f7 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:59.185107 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d2dc9-6556-49db-b8fc-ce81c27c67f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:59.203776 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-717d2dc9-6556-49db-b8fc-ce81c27c67f7 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:29:59.204951 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-717d2dc9-6556-49db-b8fc-ce81c27c67f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:29:59.269925 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 644/1303] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:29:59 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 04:29:59.292487 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c402f6-c4d2-4087-9d9a-27e273a35fee None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:59.385221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:59.391967 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c402f6-c4d2-4087-9d9a-27e273a35fee None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:59.424578 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00017af-01c3-4030-a55d-320934e49b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:59.425206 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00017af-01c3-4030-a55d-320934e49b43 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:59.425487 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00017af-01c3-4030-a55d-320934e49b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:59.425786 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00017af-01c3-4030-a55d-320934e49b43 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:59.426418 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00017af-01c3-4030-a55d-320934e49b43 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:59.428503 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00017af-01c3-4030-a55d-320934e49b43 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:59.437088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-d00017af-01c3-4030-a55d-320934e49b43 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 04:29:59.437088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:59.437088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 04:29:59.437088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 04:29:59.437088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 04:29:59.437088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 04:29:59.437088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 04:29:59.437088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:29:59.437088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 04:29:59.437088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:29:59.437088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 04:29:59.437088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:29:59.437088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 04:29:59.437088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 04:29:59.437088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 04:29:59.437088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 04:29:59.437088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 04:29:59.438268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 04:29:59.438268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 04:29:59.438268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 04:29:59.438268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 04:29:59.438268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 04:29:59.438268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 04:29:59.438268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 04:29:59.438268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 04:29:59.438268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 04:29:59.438268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 04:29:59.438268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 04:29:59.438268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 04:29:59.438268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 04:29:59.438268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 04:29:59.438268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 04:29:59.438268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 04:29:59.439137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 04:29:59.439137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 04 04:29:59.439137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 04:29:59.439137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 04 04:29:59.439137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 04:29:59.439137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 04:29:59.439137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 04:29:59.439137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 04 04:29:59.439137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 04:29:59.439137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 04 04:29:59.439137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 04:29:59.439137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 04:29:59.439137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 04:29:59.439137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 04 04:29:59.439137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 660/1304] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Aug 4 04:29:59 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 04:29:59.444362 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d2dc9-6556-49db-b8fc-ce81c27c67f7 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:59.519504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:29:59.542706 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d2dc9-6556-49db-b8fc-ce81c27c67f7 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:59.577120 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32568e4d-b831-4c7c-88ba-2c6c8f6cea75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:59.577770 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32568e4d-b831-4c7c-88ba-2c6c8f6cea75 None admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:59.578056 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32568e4d-b831-4c7c-88ba-2c6c8f6cea75 None admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:59.578332 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32568e4d-b831-4c7c-88ba-2c6c8f6cea75 None admin] PATH_INFO: `/v3/services` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:59.579006 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32568e4d-b831-4c7c-88ba-2c6c8f6cea75 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:29:59.579308 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32568e4d-b831-4c7c-88ba-2c6c8f6cea75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:59.580693 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32568e4d-b831-4c7c-88ba-2c6c8f6cea75 None admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:59.585864 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 645/1305] 10.222.0.47 () {56 vars in 1116 bytes} [Thu Aug 4 04:29:59 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:29:59.591577 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00017af-01c3-4030-a55d-320934e49b43 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:29:59.687749 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00017af-01c3-4030-a55d-320934e49b43 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:29:59.720754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39100e-cbe5-45fe-8419-2f296ca011e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:29:59.721351 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39100e-cbe5-45fe-8419-2f296ca011e4 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:29:59.721675 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39100e-cbe5-45fe-8419-2f296ca011e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:29:59.721932 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39100e-cbe5-45fe-8419-2f296ca011e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:29:59.722627 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd39100e-cbe5-45fe-8419-2f296ca011e4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:29:59.724001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd39100e-cbe5-45fe-8419-2f296ca011e4 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:29:59.743886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 661/1306] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:29:59 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:30:09.583371 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=30 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:30:09.739323 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:03.963614 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262d604-f799-481d-ad84-6f3f75dab39b None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:03.964149 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262d604-f799-481d-ad84-6f3f75dab39b None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:03.964149 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262d604-f799-481d-ad84-6f3f75dab39b None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:03.965433 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 646/1307] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:31:03.971258 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1821702-3b47-4266-b905-28243777d3b2 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:03.971810 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1821702-3b47-4266-b905-28243777d3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:03.971810 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1821702-3b47-4266-b905-28243777d3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:04.049618 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-d1821702-3b47-4266-b905-28243777d3b2 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:31:04.051473 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d1821702-3b47-4266-b905-28243777d3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:04.182237 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 662/1308] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:31:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:31:04.219428 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808a241-ac15-40d6-a8c4-c1d979eb7a1c None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:04.219615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808a241-ac15-40d6-a8c4-c1d979eb7a1c None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:04.219895 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808a241-ac15-40d6-a8c4-c1d979eb7a1c None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:04.221488 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 647/1309] 10.222.0.47 () {54 vars in 965 bytes} [Thu Aug 4 04:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:31:04.228162 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7e95b-c73d-431c-acc2-2856313d8b2a None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:04.228449 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7e95b-c73d-431c-acc2-2856313d8b2a None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:04.228730 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7e95b-c73d-431c-acc2-2856313d8b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:04.318606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-44c7e95b-c73d-431c-acc2-2856313d8b2a None None] MFA Rules not processed for user `1a2a647f97fc4d369f562e8b5a4bf68c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:31:04.320559 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-44c7e95b-c73d-431c-acc2-2856313d8b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:04.351123 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:04.443172 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 663/1310] 10.222.0.47 () {58 vars in 1062 bytes} [Thu Aug 4 04:31:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:31:04.445001 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808a241-ac15-40d6-a8c4-c1d979eb7a1c None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:31:04.448055 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-a808a241-ac15-40d6-a8c4-c1d979eb7a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:04.687832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808a241-ac15-40d6-a8c4-c1d979eb7a1c None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:31:04.784278 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e031b7-2363-4f39-90a3-6e2fa7fc8783 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:31:04.784961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e031b7-2363-4f39-90a3-6e2fa7fc8783 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:04.785291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e031b7-2363-4f39-90a3-6e2fa7fc8783 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:04.786077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e031b7-2363-4f39-90a3-6e2fa7fc8783 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:04.787217 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-97e031b7-2363-4f39-90a3-6e2fa7fc8783 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:04.857349 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:04.946415 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e031b7-2363-4f39-90a3-6e2fa7fc8783 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:31:04.948474 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e031b7-2363-4f39-90a3-6e2fa7fc8783 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:31:04.976490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 648/1311] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:31:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:31:07.356712 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d9795-2607-4769-be71-7697a4704624 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:07.357212 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d9795-2607-4769-be71-7697a4704624 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:07.357212 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d9795-2607-4769-be71-7697a4704624 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:07.358851 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 664/1312] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:31:07.364194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ee68c-3bba-4149-b60d-95ba7c844a6f None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:07.364686 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ee68c-3bba-4149-b60d-95ba7c844a6f None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:07.364867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ee68c-3bba-4149-b60d-95ba7c844a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:07.463945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-d55ee68c-3bba-4149-b60d-95ba7c844a6f None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:31:07.465638 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d55ee68c-3bba-4149-b60d-95ba7c844a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:07.592610 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 649/1313] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:31:07.648106 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e12b4-f161-4eb4-bd17-7d07e9e3f304 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:07.648511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e12b4-f161-4eb4-bd17-7d07e9e3f304 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:07.648859 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e12b4-f161-4eb4-bd17-7d07e9e3f304 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:07.650311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 665/1314] 10.222.0.47 () {54 vars in 965 bytes} [Thu Aug 4 04:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:31:07.656343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37022a21-a2b5-4150-8198-c167ec64651d None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:07.656629 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37022a21-a2b5-4150-8198-c167ec64651d None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:07.656902 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37022a21-a2b5-4150-8198-c167ec64651d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:07.669506 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:07.738869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-37022a21-a2b5-4150-8198-c167ec64651d None None] MFA Rules not processed for user `1a2a647f97fc4d369f562e8b5a4bf68c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:31:07.740263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-37022a21-a2b5-4150-8198-c167ec64651d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:07.821467 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 650/1315] 10.222.0.47 () {58 vars in 1062 bytes} [Thu Aug 4 04:31:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:31:07.830623 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e12b4-f161-4eb4-bd17-7d07e9e3f304 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:31:07.832696 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b79e12b4-f161-4eb4-bd17-7d07e9e3f304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:07.936373 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e12b4-f161-4eb4-bd17-7d07e9e3f304 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:31:08.006743 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:08.009308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb52491-10d3-4d70-8384-3156a1328c6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:31:08.010180 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb52491-10d3-4d70-8384-3156a1328c6c service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:08.010659 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb52491-10d3-4d70-8384-3156a1328c6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:08.010985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb52491-10d3-4d70-8384-3156a1328c6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:08.012794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1bb52491-10d3-4d70-8384-3156a1328c6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:08.119710 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb52491-10d3-4d70-8384-3156a1328c6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:31:08.121034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb52491-10d3-4d70-8384-3156a1328c6c service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:31:08.141145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 666/1316] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:31:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:31:10.680653 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281da4e4-8799-4e11-9eea-6184def53b23 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:10.681207 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281da4e4-8799-4e11-9eea-6184def53b23 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:10.681207 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281da4e4-8799-4e11-9eea-6184def53b23 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:10.682671 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 651/1317] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:31:10.687944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482a47b-037b-4d0a-ad5b-53ac160f5024 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:10.688364 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482a47b-037b-4d0a-ad5b-53ac160f5024 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:10.688493 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482a47b-037b-4d0a-ad5b-53ac160f5024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:10.752004 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7482a47b-037b-4d0a-ad5b-53ac160f5024 None None] MFA Rules not processed for user `e82fe3500b4c40718ae00567a41063b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:31:10.753199 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7482a47b-037b-4d0a-ad5b-53ac160f5024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:10.804292 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:10.849776 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 667/1318] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:31:10 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:31:10.892165 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281da4e4-8799-4e11-9eea-6184def53b23 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:31:10.893873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-281da4e4-8799-4e11-9eea-6184def53b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:10.997860 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281da4e4-8799-4e11-9eea-6184def53b23 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:31:11.000659 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:11.072196 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b143be41-64f7-465a-8a87-799b70874212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:31:11.072730 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143be41-64f7-465a-8a87-799b70874212 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:11.073023 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143be41-64f7-465a-8a87-799b70874212 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:11.073307 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143be41-64f7-465a-8a87-799b70874212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:11.074956 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-b143be41-64f7-465a-8a87-799b70874212 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:11.183677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b143be41-64f7-465a-8a87-799b70874212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:31:11.185108 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b143be41-64f7-465a-8a87-799b70874212 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:31:11.205805 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 652/1319] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:31:10 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:31:13.804234 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73bda3-a6a7-49a9-b23b-d20ed6394612 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:13.804753 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73bda3-a6a7-49a9-b23b-d20ed6394612 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:13.804753 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73bda3-a6a7-49a9-b23b-d20ed6394612 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:13.806151 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 668/1320] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:31:13.811348 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997335e-ecab-4de6-ae7e-3af2f8b86830 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:13.811781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997335e-ecab-4de6-ae7e-3af2f8b86830 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:13.811893 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997335e-ecab-4de6-ae7e-3af2f8b86830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:13.854186 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:13.878643 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-9997335e-ecab-4de6-ae7e-3af2f8b86830 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:31:13.880725 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-9997335e-ecab-4de6-ae7e-3af2f8b86830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:13.976964 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 653/1321] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:31:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:31:14.016914 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73bda3-a6a7-49a9-b23b-d20ed6394612 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:31:14.018465 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0a73bda3-a6a7-49a9-b23b-d20ed6394612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:14.122131 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73bda3-a6a7-49a9-b23b-d20ed6394612 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:31:14.193173 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f88e5-aeec-4578-a0d3-d2578a5a87c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:31:14.193799 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f88e5-aeec-4578-a0d3-d2578a5a87c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:14.194078 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f88e5-aeec-4578-a0d3-d2578a5a87c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:14.194347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f88e5-aeec-4578-a0d3-d2578a5a87c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:14.195934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-981f88e5-aeec-4578-a0d3-d2578a5a87c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:14.203077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:14.303030 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981f88e5-aeec-4578-a0d3-d2578a5a87c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:31:14.304451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981f88e5-aeec-4578-a0d3-d2578a5a87c9 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:31:14.322278 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 669/1322] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:31:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:31:16.989225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a31327-1216-4730-936f-23dd102137bf None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:16.989806 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a31327-1216-4730-936f-23dd102137bf None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:16.989999 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a31327-1216-4730-936f-23dd102137bf None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:16.991258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 654/1323] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:31:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:31:16.996628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a46719-996b-4b85-8dd2-d0f8bec05315 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:16.997075 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a46719-996b-4b85-8dd2-d0f8bec05315 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:16.997171 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a46719-996b-4b85-8dd2-d0f8bec05315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:17.063151 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b6a46719-996b-4b85-8dd2-d0f8bec05315 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:31:17.064347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b6a46719-996b-4b85-8dd2-d0f8bec05315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:17.128597 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:17.169690 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 670/1324] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:31:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:31:17.206829 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a31327-1216-4730-936f-23dd102137bf None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:31:17.209017 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-45a31327-1216-4730-936f-23dd102137bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:17.322220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a31327-1216-4730-936f-23dd102137bf None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:31:17.335750 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:17.404849 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07432cf-dd1c-4336-ad98-fc266eb13f45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:31:17.405476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07432cf-dd1c-4336-ad98-fc266eb13f45 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:17.405789 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07432cf-dd1c-4336-ad98-fc266eb13f45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:17.406067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07432cf-dd1c-4336-ad98-fc266eb13f45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:17.407712 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d07432cf-dd1c-4336-ad98-fc266eb13f45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:17.521886 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07432cf-dd1c-4336-ad98-fc266eb13f45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:31:17.523315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07432cf-dd1c-4336-ad98-fc266eb13f45 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:31:17.541155 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 655/1325] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:31:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:31:19.603176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3a286-8d3b-4210-b50c-c1d9ef4d3b3f None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:19.603705 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3a286-8d3b-4210-b50c-c1d9ef4d3b3f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:19.603797 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3a286-8d3b-4210-b50c-c1d9ef4d3b3f None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:19.605226 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 671/1326] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:31:19.610383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46191fe7-5069-49cd-91f9-c2d81cdd3a64 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:19.610773 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46191fe7-5069-49cd-91f9-c2d81cdd3a64 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:19.610941 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46191fe7-5069-49cd-91f9-c2d81cdd3a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:19.660050 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:19.679277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-46191fe7-5069-49cd-91f9-c2d81cdd3a64 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:31:19.680403 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-46191fe7-5069-49cd-91f9-c2d81cdd3a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:19.776293 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 656/1327] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:31:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:31:19.812920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3a286-8d3b-4210-b50c-c1d9ef4d3b3f None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:31:19.814573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-69f3a286-8d3b-4210-b50c-c1d9ef4d3b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:19.920148 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3a286-8d3b-4210-b50c-c1d9ef4d3b3f None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:31:19.994060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b6956-eed1-46a2-853d-046b4130acad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:31:19.994673 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b6956-eed1-46a2-853d-046b4130acad service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:19.994951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b6956-eed1-46a2-853d-046b4130acad service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:19.995220 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b6956-eed1-46a2-853d-046b4130acad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:19.996816 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-293b6956-eed1-46a2-853d-046b4130acad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:20.013930 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:20.105352 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b6956-eed1-46a2-853d-046b4130acad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:31:20.106850 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b6956-eed1-46a2-853d-046b4130acad service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:31:20.127810 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 672/1328] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:31:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:31:22.187164 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758a1cd-755f-4ac9-bc74-bee9e656a768 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:22.187762 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758a1cd-755f-4ac9-bc74-bee9e656a768 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:22.187762 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758a1cd-755f-4ac9-bc74-bee9e656a768 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:22.188984 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 657/1329] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:31:22.194859 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d7bae-870b-4b40-92da-d32dedbdeecc None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:22.195295 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d7bae-870b-4b40-92da-d32dedbdeecc None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:22.195395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d7bae-870b-4b40-92da-d32dedbdeecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:22.264173 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-3b6d7bae-870b-4b40-92da-d32dedbdeecc None None] MFA Rules not processed for user `e82fe3500b4c40718ae00567a41063b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:31:22.265447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3b6d7bae-870b-4b40-92da-d32dedbdeecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:22.333614 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:22.366896 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 673/1330] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:31:22 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:31:22.405220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758a1cd-755f-4ac9-bc74-bee9e656a768 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:31:22.406780 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-1758a1cd-755f-4ac9-bc74-bee9e656a768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:22.517739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758a1cd-755f-4ac9-bc74-bee9e656a768 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:31:22.539043 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:22.599861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de1c2f-fc14-4af4-9633-60c7629ff886 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:31:22.600483 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de1c2f-fc14-4af4-9633-60c7629ff886 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:22.600764 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de1c2f-fc14-4af4-9633-60c7629ff886 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:22.601040 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de1c2f-fc14-4af4-9633-60c7629ff886 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:22.602761 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-12de1c2f-fc14-4af4-9633-60c7629ff886 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:22.712750 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de1c2f-fc14-4af4-9633-60c7629ff886 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:31:22.714208 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de1c2f-fc14-4af4-9633-60c7629ff886 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:31:22.732548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 658/1331] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:31:22 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:31:24.997632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baddf4a-d379-4ae2-9a8b-c3aa0181a0b9 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:24.998159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baddf4a-d379-4ae2-9a8b-c3aa0181a0b9 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:24.998159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baddf4a-d379-4ae2-9a8b-c3aa0181a0b9 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:24.999474 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 674/1332] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:31:25.005197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca963732-3947-4ebb-ade2-c71f113e16df None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:25.005656 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca963732-3947-4ebb-ade2-c71f113e16df None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:25.005776 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca963732-3947-4ebb-ade2-c71f113e16df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:25.067764 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:25.070478 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-ca963732-3947-4ebb-ade2-c71f113e16df None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:31:25.072334 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-ca963732-3947-4ebb-ade2-c71f113e16df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:25.170089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 659/1333] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:31:25.206020 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baddf4a-d379-4ae2-9a8b-c3aa0181a0b9 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:31:25.208477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2baddf4a-d379-4ae2-9a8b-c3aa0181a0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:25.310008 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baddf4a-d379-4ae2-9a8b-c3aa0181a0b9 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:31:25.382391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c4642d-8b18-4b08-a8ba-f9234e575e02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:31:25.382995 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4642d-8b18-4b08-a8ba-f9234e575e02 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:25.383268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4642d-8b18-4b08-a8ba-f9234e575e02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:25.383533 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4642d-8b18-4b08-a8ba-f9234e575e02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:25.385127 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-36c4642d-8b18-4b08-a8ba-f9234e575e02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:25.411892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:25.492292 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c4642d-8b18-4b08-a8ba-f9234e575e02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:31:25.493737 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c4642d-8b18-4b08-a8ba-f9234e575e02 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:31:25.512254 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 675/1334] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:31:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:31:27.560849 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd184db5-93a5-4a6b-9015-877e72d936c3 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:27.561365 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd184db5-93a5-4a6b-9015-877e72d936c3 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:27.561478 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd184db5-93a5-4a6b-9015-877e72d936c3 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:27.562770 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 660/1335] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:31:27.568062 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a88de-5776-46ab-bcc6-c339c1fc28ba None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:27.568507 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a88de-5776-46ab-bcc6-c339c1fc28ba None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:27.568611 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a88de-5776-46ab-bcc6-c339c1fc28ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:27.634858 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-4d8a88de-5776-46ab-bcc6-c339c1fc28ba None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:31:27.636105 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4d8a88de-5776-46ab-bcc6-c339c1fc28ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:27.709510 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:27.732101 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 676/1336] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:31:27.782462 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd184db5-93a5-4a6b-9015-877e72d936c3 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:31:27.784015 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-fd184db5-93a5-4a6b-9015-877e72d936c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:27.887428 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd184db5-93a5-4a6b-9015-877e72d936c3 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:31:27.915058 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:27.959941 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debb147-cee7-4f0e-b2e1-e048239a9125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:31:27.960552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debb147-cee7-4f0e-b2e1-e048239a9125 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:27.960827 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debb147-cee7-4f0e-b2e1-e048239a9125 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:27.961095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debb147-cee7-4f0e-b2e1-e048239a9125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:27.962742 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-1debb147-cee7-4f0e-b2e1-e048239a9125 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:28.071244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debb147-cee7-4f0e-b2e1-e048239a9125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:31:28.072703 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debb147-cee7-4f0e-b2e1-e048239a9125 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:31:28.091937 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 661/1337] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:31:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:31:30.444804 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a3326-833d-4879-915a-d83c5431f924 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:30.445343 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a3326-833d-4879-915a-d83c5431f924 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:30.445343 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a3326-833d-4879-915a-d83c5431f924 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:30.446724 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 677/1338] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:31:30.452299 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b01347-889b-4084-97d8-eb6031c0040b None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:30.452705 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b01347-889b-4084-97d8-eb6031c0040b None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:30.452868 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b01347-889b-4084-97d8-eb6031c0040b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:30.521754 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-f7b01347-889b-4084-97d8-eb6031c0040b None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:31:30.522966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-f7b01347-889b-4084-97d8-eb6031c0040b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:30.531233 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:30.621309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 662/1339] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:31:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:31:30.658475 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a3326-833d-4879-915a-d83c5431f924 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:31:30.659969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-512a3326-833d-4879-915a-d83c5431f924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:30.762277 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a3326-833d-4879-915a-d83c5431f924 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:31:30.835146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84d753-b143-4013-ac33-fe24426903b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:31:30.835752 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84d753-b143-4013-ac33-fe24426903b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:30.836034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84d753-b143-4013-ac33-fe24426903b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:30.836308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84d753-b143-4013-ac33-fe24426903b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:30.837948 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9e84d753-b143-4013-ac33-fe24426903b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:30.872779 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:30.947376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84d753-b143-4013-ac33-fe24426903b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:31:30.948848 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84d753-b143-4013-ac33-fe24426903b7 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:31:30.968748 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 678/1340] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:31:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:31:34.919612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d98b1-84e0-4224-af4c-115abc77ec11 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:34.920261 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d98b1-84e0-4224-af4c-115abc77ec11 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:34.920261 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d98b1-84e0-4224-af4c-115abc77ec11 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:34.921577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 663/1341] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:31:34.927858 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fc07d-bade-4ddd-b587-c5a2437dc0e0 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:34.928304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fc07d-bade-4ddd-b587-c5a2437dc0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:34.928425 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fc07d-bade-4ddd-b587-c5a2437dc0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:34.997132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-714fc07d-bade-4ddd-b587-c5a2437dc0e0 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:31:34.998585 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-714fc07d-bade-4ddd-b587-c5a2437dc0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:35.084786 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:35.098310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 679/1342] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:31:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:31:35.137516 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d98b1-84e0-4224-af4c-115abc77ec11 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:31:35.139205 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-702d98b1-84e0-4224-af4c-115abc77ec11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:35.248917 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d98b1-84e0-4224-af4c-115abc77ec11 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:31:35.292499 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:35.329137 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199fbd83-f035-4aac-9a86-9b3ea2c1df80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:31:35.329777 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fbd83-f035-4aac-9a86-9b3ea2c1df80 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:35.330057 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fbd83-f035-4aac-9a86-9b3ea2c1df80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:35.330325 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fbd83-f035-4aac-9a86-9b3ea2c1df80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:35.331931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-199fbd83-f035-4aac-9a86-9b3ea2c1df80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:35.450699 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199fbd83-f035-4aac-9a86-9b3ea2c1df80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:31:35.452036 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199fbd83-f035-4aac-9a86-9b3ea2c1df80 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:31:35.470627 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 664/1343] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:31:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:31:38.200381 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25fb8a-1491-407f-8657-4491cab38c4a None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:38.200903 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25fb8a-1491-407f-8657-4491cab38c4a None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:38.200903 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25fb8a-1491-407f-8657-4491cab38c4a None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:38.202207 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 680/1344] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:31:38.207829 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973b8fd-7ef1-405a-a90d-97b19c9fed0e None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:38.208356 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973b8fd-7ef1-405a-a90d-97b19c9fed0e None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:38.208459 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973b8fd-7ef1-405a-a90d-97b19c9fed0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:38.283493 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-7973b8fd-7ef1-405a-a90d-97b19c9fed0e None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:31:38.284682 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-7973b8fd-7ef1-405a-a90d-97b19c9fed0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:38.304613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:38.389694 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 665/1345] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:31:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:31:38.427636 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25fb8a-1491-407f-8657-4491cab38c4a None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:31:38.429169 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0b25fb8a-1491-407f-8657-4491cab38c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:38.534027 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25fb8a-1491-407f-8657-4491cab38c4a None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:31:38.614665 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ce161-a3dc-4964-be98-93f93c454ea1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:31:38.615278 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ce161-a3dc-4964-be98-93f93c454ea1 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:38.615557 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ce161-a3dc-4964-be98-93f93c454ea1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:38.615825 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ce161-a3dc-4964-be98-93f93c454ea1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:38.617429 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f74ce161-a3dc-4964-be98-93f93c454ea1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:38.661401 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:38.728949 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ce161-a3dc-4964-be98-93f93c454ea1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:31:38.730414 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ce161-a3dc-4964-be98-93f93c454ea1 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:31:38.749309 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 681/1346] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:31:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:31:42.819580 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1520f485-4968-4334-a662-bbdf257c9fe0 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:42.820150 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1520f485-4968-4334-a662-bbdf257c9fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:42.820150 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1520f485-4968-4334-a662-bbdf257c9fe0 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:42.821489 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 666/1347] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:31:42.826593 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154eaef-7439-4d6e-9ea6-0cce605ad7cd None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:42.827009 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154eaef-7439-4d6e-9ea6-0cce605ad7cd None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:42.827141 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154eaef-7439-4d6e-9ea6-0cce605ad7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:42.897972 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b154eaef-7439-4d6e-9ea6-0cce605ad7cd None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:31:42.899245 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b154eaef-7439-4d6e-9ea6-0cce605ad7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:43.000943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:43.004117 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 682/1348] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:31:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:31:43.044956 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1520f485-4968-4334-a662-bbdf257c9fe0 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:31:43.046548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-1520f485-4968-4334-a662-bbdf257c9fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:43.179677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1520f485-4968-4334-a662-bbdf257c9fe0 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:31:43.233930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:43.262530 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc00eba-3241-41f3-b2ca-426435330bd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:31:43.263154 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc00eba-3241-41f3-b2ca-426435330bd4 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:43.263439 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc00eba-3241-41f3-b2ca-426435330bd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:43.263717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc00eba-3241-41f3-b2ca-426435330bd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:43.265362 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5bc00eba-3241-41f3-b2ca-426435330bd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:43.385229 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc00eba-3241-41f3-b2ca-426435330bd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:31:43.386735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc00eba-3241-41f3-b2ca-426435330bd4 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:31:43.406100 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 667/1349] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:31:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:31:48.469795 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363586a7-1fb1-4680-ba7a-4563c7581ead None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:48.469795 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363586a7-1fb1-4680-ba7a-4563c7581ead None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:48.469795 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363586a7-1fb1-4680-ba7a-4563c7581ead None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:48.471040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 683/1350] 10.222.0.47 () {54 vars in 921 bytes} [Thu Aug 4 04:31:48 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:31:48.482688 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3de98b-e91f-44b5-8d42-d3d366e62812 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:48.483206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3de98b-e91f-44b5-8d42-d3d366e62812 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:48.483206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3de98b-e91f-44b5-8d42-d3d366e62812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:48.607195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-9c3de98b-e91f-44b5-8d42-d3d366e62812 None None] MFA Rules not processed for user `350f72f740f749f68175c1c4e8ae9d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:31:48.608680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-9c3de98b-e91f-44b5-8d42-d3d366e62812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:48.633142 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:48.759308 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 668/1351] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Aug 4 04:31:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:31:52.800427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46be3b-8cdc-41d9-bd97-d7ef94e9486e None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:52.801392 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46be3b-8cdc-41d9-bd97-d7ef94e9486e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:52.801851 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46be3b-8cdc-41d9-bd97-d7ef94e9486e None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:52.803650 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 684/1352] 10.222.0.47 () {54 vars in 921 bytes} [Thu Aug 4 04:31:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:31:52.809895 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f3fe2-aafa-4fe3-ae0a-aeca380811a7 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:52.810760 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f3fe2-aafa-4fe3-ae0a-aeca380811a7 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:52.811035 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f3fe2-aafa-4fe3-ae0a-aeca380811a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:52.899935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-116f3fe2-aafa-4fe3-ae0a-aeca380811a7 None None] MFA Rules not processed for user `350f72f740f749f68175c1c4e8ae9d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:31:52.901130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-116f3fe2-aafa-4fe3-ae0a-aeca380811a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:52.984650 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:53.024877 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 669/1353] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Aug 4 04:31:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:31:54.625704 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044ab5a-af78-4829-a1b2-786624110d91 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:54.626258 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044ab5a-af78-4829-a1b2-786624110d91 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:54.626351 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044ab5a-af78-4829-a1b2-786624110d91 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:54.627724 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 685/1354] 10.222.0.47 () {54 vars in 921 bytes} [Thu Aug 4 04:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:31:54.634251 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7684d4-17d4-4a6a-8f2c-18da705c400d None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:54.634681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7684d4-17d4-4a6a-8f2c-18da705c400d None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:54.634812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7684d4-17d4-4a6a-8f2c-18da705c400d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:54.713271 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-2b7684d4-17d4-4a6a-8f2c-18da705c400d None None] MFA Rules not processed for user `350f72f740f749f68175c1c4e8ae9d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:31:54.714522 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-2b7684d4-17d4-4a6a-8f2c-18da705c400d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:54.821762 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 670/1355] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Aug 4 04:31:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:31:58.319804 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1f31d-a215-4b3c-9254-e8b872a9458f None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:58.320675 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1f31d-a215-4b3c-9254-e8b872a9458f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:58.321089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1f31d-a215-4b3c-9254-e8b872a9458f None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:58.327494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 686/1356] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:31:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:31:58.329307 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb7fd8-f222-4ea1-aadb-fca1a186d034 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:58.329859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb7fd8-f222-4ea1-aadb-fca1a186d034 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:58.329859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb7fd8-f222-4ea1-aadb-fca1a186d034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:58.395007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:58.433033 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-4ccb7fd8-f222-4ea1-aadb-fca1a186d034 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:31:58.434265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-4ccb7fd8-f222-4ea1-aadb-fca1a186d034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:58.548023 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 671/1357] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:31:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:31:58.605667 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1f9b1-e564-4992-81ea-e0b336281dc8 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:58.606372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1f9b1-e564-4992-81ea-e0b336281dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:58.606936 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1f9b1-e564-4992-81ea-e0b336281dc8 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:58.610119 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 687/1358] 10.222.0.47 () {54 vars in 962 bytes} [Thu Aug 4 04:31:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:31:58.622032 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd3b63-d950-4d16-b34e-115f990fd9a3 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:58.622967 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd3b63-d950-4d16-b34e-115f990fd9a3 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:58.623968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd3b63-d950-4d16-b34e-115f990fd9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:58.712310 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-7fdd3b63-d950-4d16-b34e-115f990fd9a3 None None] MFA Rules not processed for user `caa2901451aa4554bdfcac1bddbc620a`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:31:58.713700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-7fdd3b63-d950-4d16-b34e-115f990fd9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:58.766020 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:58.847452 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 672/1359] 10.222.0.47 () {58 vars in 1059 bytes} [Thu Aug 4 04:31:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:31:58.856401 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1f9b1-e564-4992-81ea-e0b336281dc8 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:31:58.857899 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b0f1f9b1-e564-4992-81ea-e0b336281dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:59.012855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1f9b1-e564-4992-81ea-e0b336281dc8 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:31:59.093697 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:31:59.138002 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e94960-b33e-4a58-8b12-83b9ef4138df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:31:59.138676 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e94960-b33e-4a58-8b12-83b9ef4138df service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:31:59.138965 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e94960-b33e-4a58-8b12-83b9ef4138df service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:31:59.139284 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e94960-b33e-4a58-8b12-83b9ef4138df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:31:59.140975 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d1e94960-b33e-4a58-8b12-83b9ef4138df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:31:59.238530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e94960-b33e-4a58-8b12-83b9ef4138df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:31:59.240153 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e94960-b33e-4a58-8b12-83b9ef4138df service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:31:59.259448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 688/1360] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:31:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:32:01.535189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d410d2f-deac-43ae-8f8d-1d242e25114c None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:01.536052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d410d2f-deac-43ae-8f8d-1d242e25114c None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:01.536335 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d410d2f-deac-43ae-8f8d-1d242e25114c None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:01.537971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 673/1361] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:32:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:32:01.543991 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34ac0c-c617-4de5-9100-b901845438f8 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:01.544454 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34ac0c-c617-4de5-9100-b901845438f8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:01.544532 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34ac0c-c617-4de5-9100-b901845438f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:01.617990 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-da34ac0c-c617-4de5-9100-b901845438f8 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:01.619194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-da34ac0c-c617-4de5-9100-b901845438f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:01.660484 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:01.726245 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 689/1362] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:32:01.774453 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d410d2f-deac-43ae-8f8d-1d242e25114c None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:32:01.776375 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-1d410d2f-deac-43ae-8f8d-1d242e25114c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:01.957107 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d410d2f-deac-43ae-8f8d-1d242e25114c None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:32:01.967174 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:02.118921 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9466dff-6b08-4930-9556-cf19bf03e7f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:32:02.119539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9466dff-6b08-4930-9556-cf19bf03e7f5 service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:02.119815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9466dff-6b08-4930-9556-cf19bf03e7f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:02.120082 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9466dff-6b08-4930-9556-cf19bf03e7f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:02.121736 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c9466dff-6b08-4930-9556-cf19bf03e7f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:02.215283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9466dff-6b08-4930-9556-cf19bf03e7f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:32:02.216862 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9466dff-6b08-4930-9556-cf19bf03e7f5 service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:32:02.236099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 674/1363] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:32:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:32:04.634861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08138225-5ee3-4b1e-b445-84bd7921a8af None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:04.635402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08138225-5ee3-4b1e-b445-84bd7921a8af None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:04.635502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08138225-5ee3-4b1e-b445-84bd7921a8af None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:04.636880 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 690/1364] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:32:04.642422 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b0148-7f60-4b1e-8fa0-9cec7fc8b85f None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:04.642860 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b0148-7f60-4b1e-8fa0-9cec7fc8b85f None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:04.643081 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b0148-7f60-4b1e-8fa0-9cec7fc8b85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:04.662904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:04.731976 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-ec2b0148-7f60-4b1e-8fa0-9cec7fc8b85f None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:04.733309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-ec2b0148-7f60-4b1e-8fa0-9cec7fc8b85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:04.840930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 675/1365] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:32:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:32:04.888384 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eccfab-da16-4152-841e-d0a704453766 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:04.888885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eccfab-da16-4152-841e-d0a704453766 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:04.889595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eccfab-da16-4152-841e-d0a704453766 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:04.891076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 691/1366] 10.222.0.47 () {54 vars in 962 bytes} [Thu Aug 4 04:32:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:32:04.897455 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd854f76-04c8-4963-9fab-be8d29d96bda None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:04.897803 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd854f76-04c8-4963-9fab-be8d29d96bda None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:04.898080 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd854f76-04c8-4963-9fab-be8d29d96bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:04.981175 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-bd854f76-04c8-4963-9fab-be8d29d96bda None None] MFA Rules not processed for user `caa2901451aa4554bdfcac1bddbc620a`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:04.982498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-bd854f76-04c8-4963-9fab-be8d29d96bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:05.022446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:05.117113 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 676/1367] 10.222.0.47 () {58 vars in 1059 bytes} [Thu Aug 4 04:32:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:32:05.122367 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eccfab-da16-4152-841e-d0a704453766 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:32:05.123976 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-19eccfab-da16-4152-841e-d0a704453766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:05.288267 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eccfab-da16-4152-841e-d0a704453766 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:32:05.310681 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:05.411439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1eef8-a60a-479b-84cf-9732740caf11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:32:05.411966 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1eef8-a60a-479b-84cf-9732740caf11 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:05.412243 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1eef8-a60a-479b-84cf-9732740caf11 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:05.412514 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1eef8-a60a-479b-84cf-9732740caf11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:05.414139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-78f1eef8-a60a-479b-84cf-9732740caf11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:05.516822 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1eef8-a60a-479b-84cf-9732740caf11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:32:05.518828 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1eef8-a60a-479b-84cf-9732740caf11 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:32:05.539540 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 692/1368] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:32:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:32:06.555788 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1853a-ecf5-49ff-899d-73a5f9338df6 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:06.556374 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1853a-ecf5-49ff-899d-73a5f9338df6 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:06.556374 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1853a-ecf5-49ff-899d-73a5f9338df6 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:06.557875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 677/1369] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:32:06.562986 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06976ca8-41bd-4436-853d-39b6e8854396 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:06.563543 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06976ca8-41bd-4436-853d-39b6e8854396 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:06.563543 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06976ca8-41bd-4436-853d-39b6e8854396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:06.597260 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:06.634599 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-06976ca8-41bd-4436-853d-39b6e8854396 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:06.635814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-06976ca8-41bd-4436-853d-39b6e8854396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:06.743211 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 693/1370] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:32:07.088013 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdd7be-227a-4b6e-9faf-c83f16fc519e None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:07.088191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdd7be-227a-4b6e-9faf-c83f16fc519e None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:07.088562 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdd7be-227a-4b6e-9faf-c83f16fc519e None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:07.090046 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 678/1371] 10.222.0.47 () {54 vars in 964 bytes} [Thu Aug 4 04:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:32:07.096018 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4639c5-fe51-4c48-ba68-f78002a9b6cd None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:07.096018 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4639c5-fe51-4c48-ba68-f78002a9b6cd None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:07.096018 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4639c5-fe51-4c48-ba68-f78002a9b6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:07.177994 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ff4639c5-fe51-4c48-ba68-f78002a9b6cd None None] MFA Rules not processed for user `8b1d67b28dca49ca9cb0ff28cad62aa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:07.179666 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ff4639c5-fe51-4c48-ba68-f78002a9b6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:07.230100 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:07.261852 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 694/1372] 10.222.0.47 () {58 vars in 1061 bytes} [Thu Aug 4 04:32:07 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:32:07.269969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdd7be-227a-4b6e-9faf-c83f16fc519e None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:32:07.271779 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-bcfdd7be-227a-4b6e-9faf-c83f16fc519e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:07.386610 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdd7be-227a-4b6e-9faf-c83f16fc519e None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:32:07.429620 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:07.466812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09954756-c524-4382-947e-b68615f08f09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:32:07.467441 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09954756-c524-4382-947e-b68615f08f09 service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:07.467726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09954756-c524-4382-947e-b68615f08f09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:07.467993 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09954756-c524-4382-947e-b68615f08f09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:07.469644 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-09954756-c524-4382-947e-b68615f08f09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:07.595092 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09954756-c524-4382-947e-b68615f08f09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:32:07.596646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09954756-c524-4382-947e-b68615f08f09 service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:32:07.616342 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 679/1373] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:32:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:32:07.658330 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c70a6-7dbb-4667-bef8-33f8f0991e08 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:07.658900 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c70a6-7dbb-4667-bef8-33f8f0991e08 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:07.659138 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c70a6-7dbb-4667-bef8-33f8f0991e08 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:07.661499 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 695/1374] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:32:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:32:07.666924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d704d0c-3de0-4656-aace-ba07cf302d1f None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:07.667065 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d704d0c-3de0-4656-aace-ba07cf302d1f None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:07.667339 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d704d0c-3de0-4656-aace-ba07cf302d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:07.747846 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-4d704d0c-3de0-4656-aace-ba07cf302d1f None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:07.749271 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-4d704d0c-3de0-4656-aace-ba07cf302d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:07.773585 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:07.878599 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 680/1375] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:32:07.915015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c70a6-7dbb-4667-bef8-33f8f0991e08 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:32:07.917101 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e09c70a6-7dbb-4667-bef8-33f8f0991e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:08.108503 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c70a6-7dbb-4667-bef8-33f8f0991e08 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:32:08.174542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:08.246023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e41f02-46b9-416d-b860-0818ffe998ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:32:08.246983 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e41f02-46b9-416d-b860-0818ffe998ce service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:08.247325 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e41f02-46b9-416d-b860-0818ffe998ce service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:08.247842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e41f02-46b9-416d-b860-0818ffe998ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:08.249646 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f6e41f02-46b9-416d-b860-0818ffe998ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:08.352134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e41f02-46b9-416d-b860-0818ffe998ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:32:08.353770 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e41f02-46b9-416d-b860-0818ffe998ce service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:32:08.375182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 696/1376] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:32:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:32:10.497361 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38da8e-2802-4318-b9f2-a140a81edb9f None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:10.497904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38da8e-2802-4318-b9f2-a140a81edb9f None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:10.497904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38da8e-2802-4318-b9f2-a140a81edb9f None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:10.500421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 681/1377] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:32:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:32:10.505494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47d555-c4b7-46d2-ba5d-536ca5461701 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:10.506005 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47d555-c4b7-46d2-ba5d-536ca5461701 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:10.506527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47d555-c4b7-46d2-ba5d-536ca5461701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:10.583518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-6b47d555-c4b7-46d2-ba5d-536ca5461701 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:10.584744 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6b47d555-c4b7-46d2-ba5d-536ca5461701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:10.591797 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:10.693588 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 697/1378] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:32:10.736221 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38da8e-2802-4318-b9f2-a140a81edb9f None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:32:10.737813 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-0d38da8e-2802-4318-b9f2-a140a81edb9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:10.940599 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38da8e-2802-4318-b9f2-a140a81edb9f None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:32:10.945292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:11.071458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d815a21-8ba9-4071-bcee-bfa887cd1dbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:32:11.072098 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d815a21-8ba9-4071-bcee-bfa887cd1dbb service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:11.072355 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d815a21-8ba9-4071-bcee-bfa887cd1dbb service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:11.072620 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d815a21-8ba9-4071-bcee-bfa887cd1dbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:11.074286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-3d815a21-8ba9-4071-bcee-bfa887cd1dbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:11.171883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d815a21-8ba9-4071-bcee-bfa887cd1dbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:32:11.173666 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d815a21-8ba9-4071-bcee-bfa887cd1dbb service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:32:11.194508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 682/1379] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:32:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:32:13.357093 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c19a0-bf89-4432-8611-0c064bddcc57 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:13.357636 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c19a0-bf89-4432-8611-0c064bddcc57 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:13.357636 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c19a0-bf89-4432-8611-0c064bddcc57 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:13.358996 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 698/1380] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:32:13.365532 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b101b3-9e45-4211-9e50-01cd16d8f3ff None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:13.366010 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b101b3-9e45-4211-9e50-01cd16d8f3ff None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:13.366229 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b101b3-9e45-4211-9e50-01cd16d8f3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:13.379051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:13.443665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-49b101b3-9e45-4211-9e50-01cd16d8f3ff None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:13.444892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-49b101b3-9e45-4211-9e50-01cd16d8f3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:13.589596 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 683/1381] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:32:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:32:13.637301 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c19a0-bf89-4432-8611-0c064bddcc57 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:32:13.638697 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-519c19a0-bf89-4432-8611-0c064bddcc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:13.832774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:13.845579 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c19a0-bf89-4432-8611-0c064bddcc57 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:32:14.010528 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b79834-d9e2-4ee3-8070-a0eda9eb1b1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:32:14.011472 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b79834-d9e2-4ee3-8070-a0eda9eb1b1a service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:14.011942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b79834-d9e2-4ee3-8070-a0eda9eb1b1a service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:14.012400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b79834-d9e2-4ee3-8070-a0eda9eb1b1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:14.014311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-96b79834-d9e2-4ee3-8070-a0eda9eb1b1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:14.128374 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b79834-d9e2-4ee3-8070-a0eda9eb1b1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:32:14.130704 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b79834-d9e2-4ee3-8070-a0eda9eb1b1a service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:32:14.142636 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:14.154099 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 699/1382] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:32:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:32:16.323992 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab1959-1b2b-4ff1-8cea-4e42bc65674c None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:16.324508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab1959-1b2b-4ff1-8cea-4e42bc65674c None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:16.324607 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab1959-1b2b-4ff1-8cea-4e42bc65674c None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:16.326068 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 684/1383] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:32:16.331616 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2106b7-9baf-4159-bba1-c1eda815bc76 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:16.332450 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2106b7-9baf-4159-bba1-c1eda815bc76 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:16.332450 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2106b7-9baf-4159-bba1-c1eda815bc76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:16.419389 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-2b2106b7-9baf-4159-bba1-c1eda815bc76 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:16.420451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2b2106b7-9baf-4159-bba1-c1eda815bc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:16.557052 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 700/1384] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:32:16.604361 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab1959-1b2b-4ff1-8cea-4e42bc65674c None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:32:16.605925 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-96ab1959-1b2b-4ff1-8cea-4e42bc65674c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:16.704758 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:16.778138 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab1959-1b2b-4ff1-8cea-4e42bc65674c None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:32:16.943018 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc65b16-1c77-4232-8422-02b62d763acf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:32:16.943765 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc65b16-1c77-4232-8422-02b62d763acf service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:16.944110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc65b16-1c77-4232-8422-02b62d763acf service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:16.944482 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc65b16-1c77-4232-8422-02b62d763acf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:16.946527 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-2dc65b16-1c77-4232-8422-02b62d763acf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:17.006633 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:17.057495 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc65b16-1c77-4232-8422-02b62d763acf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:32:17.059235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc65b16-1c77-4232-8422-02b62d763acf service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:32:17.079564 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 685/1385] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:32:19.219392 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758cef3-bbe1-4717-a8f0-e292a994b910 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:19.220042 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758cef3-bbe1-4717-a8f0-e292a994b910 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:19.220042 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758cef3-bbe1-4717-a8f0-e292a994b910 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:19.221317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 701/1386] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:32:19.227540 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a05f11-3b53-454d-8339-9d9215ef47cf None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:19.228039 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a05f11-3b53-454d-8339-9d9215ef47cf None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:19.228245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a05f11-3b53-454d-8339-9d9215ef47cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:19.298427 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-d1a05f11-3b53-454d-8339-9d9215ef47cf None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:19.299650 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d1a05f11-3b53-454d-8339-9d9215ef47cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:19.401701 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 686/1387] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:32:19.447594 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758cef3-bbe1-4717-a8f0-e292a994b910 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:32:19.449215 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-5758cef3-bbe1-4717-a8f0-e292a994b910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:19.554093 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:19.632793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758cef3-bbe1-4717-a8f0-e292a994b910 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:32:19.750110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d489c9b-24dd-48a6-914b-9f648ec4965d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:32:19.750647 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d489c9b-24dd-48a6-914b-9f648ec4965d service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:19.750925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d489c9b-24dd-48a6-914b-9f648ec4965d service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:19.751197 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d489c9b-24dd-48a6-914b-9f648ec4965d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:19.752816 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6d489c9b-24dd-48a6-914b-9f648ec4965d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:19.814379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:19.888641 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d489c9b-24dd-48a6-914b-9f648ec4965d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:32:19.890375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d489c9b-24dd-48a6-914b-9f648ec4965d service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:32:19.909379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 702/1388] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:32:21.971035 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c6657-8f98-4eca-b6bb-41e609da9890 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:21.971567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c6657-8f98-4eca-b6bb-41e609da9890 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:21.971567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c6657-8f98-4eca-b6bb-41e609da9890 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:21.973043 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 687/1389] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:32:21.978396 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b12b5-378f-412f-b9cd-d2164bf194d7 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:21.978852 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b12b5-378f-412f-b9cd-d2164bf194d7 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:21.978953 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b12b5-378f-412f-b9cd-d2164bf194d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:22.044730 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-e07b12b5-378f-412f-b9cd-d2164bf194d7 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:22.045966 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e07b12b5-378f-412f-b9cd-d2164bf194d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:22.143752 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 703/1390] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:32:22.190351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c6657-8f98-4eca-b6bb-41e609da9890 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:32:22.192042 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-bf5c6657-8f98-4eca-b6bb-41e609da9890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:22.224919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:22.354882 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c6657-8f98-4eca-b6bb-41e609da9890 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:32:22.581168 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:22.593744 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ae47c-a139-4227-8302-6fe532f00ec4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:32:22.594363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ae47c-a139-4227-8302-6fe532f00ec4 service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:22.594645 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ae47c-a139-4227-8302-6fe532f00ec4 service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:22.594916 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ae47c-a139-4227-8302-6fe532f00ec4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:22.596530 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-ac6ae47c-a139-4227-8302-6fe532f00ec4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:22.687450 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ae47c-a139-4227-8302-6fe532f00ec4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:32:22.689035 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ae47c-a139-4227-8302-6fe532f00ec4 service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:32:22.708184 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 688/1391] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:32:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:32:23.371235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be330b2b-e8c5-4b01-8816-00517a43dfaf None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:23.371813 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be330b2b-e8c5-4b01-8816-00517a43dfaf None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:23.371938 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be330b2b-e8c5-4b01-8816-00517a43dfaf None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:23.373581 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 704/1392] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:32:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:32:23.379585 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3c4b4-39bd-4a22-ad25-f24fe64667bc None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:23.380113 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3c4b4-39bd-4a22-ad25-f24fe64667bc None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:23.380113 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3c4b4-39bd-4a22-ad25-f24fe64667bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:23.446382 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-07a3c4b4-39bd-4a22-ad25-f24fe64667bc None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:23.447573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-07a3c4b4-39bd-4a22-ad25-f24fe64667bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:23.512541 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:23.552345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 689/1393] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:32:23.608175 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ce591-5802-458c-999b-9ab757f2f937 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:23.608458 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ce591-5802-458c-999b-9ab757f2f937 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:23.608752 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ce591-5802-458c-999b-9ab757f2f937 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:23.610826 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 705/1394] 10.222.0.47 () {54 vars in 966 bytes} [Thu Aug 4 04:32:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:32:23.616567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0786058-a4ee-44aa-b38f-f052dc33f82e None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:23.616995 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0786058-a4ee-44aa-b38f-f052dc33f82e None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:23.617096 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0786058-a4ee-44aa-b38f-f052dc33f82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:23.693172 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-d0786058-a4ee-44aa-b38f-f052dc33f82e None None] MFA Rules not processed for user `b77a955866c54a43a7aaa271fefdec57`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:23.694327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d0786058-a4ee-44aa-b38f-f052dc33f82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:23.770659 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 690/1395] 10.222.0.47 () {58 vars in 1063 bytes} [Thu Aug 4 04:32:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:32:23.778434 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ce591-5802-458c-999b-9ab757f2f937 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:32:23.780009 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ba7ce591-5802-458c-999b-9ab757f2f937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:23.793951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:23.895621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ce591-5802-458c-999b-9ab757f2f937 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:32:23.969024 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48cee82-3cb3-47ca-b869-c57291950af4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b77a955866c54a43a7aaa271fefdec57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:32:23.969578 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48cee82-3cb3-47ca-b869-c57291950af4 service glance] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:23.969865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48cee82-3cb3-47ca-b869-c57291950af4 service glance] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:23.970138 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48cee82-3cb3-47ca-b869-c57291950af4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:23.971742 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e48cee82-3cb3-47ca-b869-c57291950af4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:24.039156 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:24.080675 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48cee82-3cb3-47ca-b869-c57291950af4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:32:24.082898 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48cee82-3cb3-47ca-b869-c57291950af4 service glance] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:32:24.102323 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 706/1396] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:32:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:32:24.767305 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a13a99-ac74-467c-8383-1cc6e2b305be None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:24.767826 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a13a99-ac74-467c-8383-1cc6e2b305be None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:24.767826 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a13a99-ac74-467c-8383-1cc6e2b305be None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:24.769254 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 691/1397] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:32:24.775129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb8863-71cb-427e-8741-9f0ac049c2f0 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:24.775662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb8863-71cb-427e-8741-9f0ac049c2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:24.775662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb8863-71cb-427e-8741-9f0ac049c2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:24.848215 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57d624-9c99-4d25-abd8-6aa4c629acb2 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:24.848526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57d624-9c99-4d25-abd8-6aa4c629acb2 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:24.848764 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57d624-9c99-4d25-abd8-6aa4c629acb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:24.855542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-2ffb8863-71cb-427e-8741-9f0ac049c2f0 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:24.856760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2ffb8863-71cb-427e-8741-9f0ac049c2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:24.915124 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-fe57d624-9c99-4d25-abd8-6aa4c629acb2 None None] MFA Rules not processed for user `3a721708c7e443e799349add27de8f84`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:24.916273 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-fe57d624-9c99-4d25-abd8-6aa4c629acb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:24.924848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:24.956442 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 707/1398] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:32:25.001448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb8863-71cb-427e-8741-9f0ac049c2f0 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:32:25.002919 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2ffb8863-71cb-427e-8741-9f0ac049c2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:25.007527 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 692/1399] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Aug 4 04:32:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:32:25.030981 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:25.055851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068d21c-0952-47c4-b7f2-da379439b2de None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:25.056143 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068d21c-0952-47c4-b7f2-da379439b2de None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:25.056421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068d21c-0952-47c4-b7f2-da379439b2de None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:25.057831 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 693/1400] 10.222.0.47 () {54 vars in 943 bytes} [Thu Aug 4 04:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:32:25.063899 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366af79-e0a6-4d00-b8fc-728cc576f4a9 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:25.064219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366af79-e0a6-4d00-b8fc-728cc576f4a9 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:25.064497 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366af79-e0a6-4d00-b8fc-728cc576f4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:25.121384 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb8863-71cb-427e-8741-9f0ac049c2f0 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:32:25.141874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-2366af79-e0a6-4d00-b8fc-728cc576f4a9 None None] MFA Rules not processed for user `b1e539006166434790823b839bd745d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:25.143097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-2366af79-e0a6-4d00-b8fc-728cc576f4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:25.230366 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:25.236072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5bda0f-8365-4a4b-b779-087c59dd0119 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:32:25.236693 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bda0f-8365-4a4b-b779-087c59dd0119 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:25.236972 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bda0f-8365-4a4b-b779-087c59dd0119 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:25.237243 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bda0f-8365-4a4b-b779-087c59dd0119 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:25.239755 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ce5bda0f-8365-4a4b-b779-087c59dd0119 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:25.256135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 694/1401] 10.222.0.47 () {58 vars in 1040 bytes} [Thu Aug 4 04:32:25 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:32:25.258619 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:25.265618 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366af79-e0a6-4d00-b8fc-728cc576f4a9 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:32:25.276407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366af79-e0a6-4d00-b8fc-728cc576f4a9 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:32:25.336726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5bda0f-8365-4a4b-b779-087c59dd0119 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:32:25.338429 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5bda0f-8365-4a4b-b779-087c59dd0119 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:32:25.357516 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 708/1402] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:32:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:32:25.386316 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de1940-ef14-4a38-a689-66983fe085ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1e539006166434790823b839bd745d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:32:25.386934 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de1940-ef14-4a38-a689-66983fe085ab service swift] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:25.387240 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de1940-ef14-4a38-a689-66983fe085ab service swift] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:25.387480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de1940-ef14-4a38-a689-66983fe085ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:25.389232 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-42de1940-ef14-4a38-a689-66983fe085ab service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:25.484263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de1940-ef14-4a38-a689-66983fe085ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:32:25.485908 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de1940-ef14-4a38-a689-66983fe085ab service swift] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:32:25.496131 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 695/1403] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Aug 4 04:32:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 04:32:27.436472 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9860e0d-2639-499b-bfee-ff830a0d3b13 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:27.437030 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9860e0d-2639-499b-bfee-ff830a0d3b13 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:27.437030 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9860e0d-2639-499b-bfee-ff830a0d3b13 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:27.438355 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 709/1404] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:32:27.444555 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258daa59-dfcf-4a00-a639-43400c29baa3 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:27.445025 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258daa59-dfcf-4a00-a639-43400c29baa3 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:27.445226 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258daa59-dfcf-4a00-a639-43400c29baa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:27.457010 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:27.569340 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-258daa59-dfcf-4a00-a639-43400c29baa3 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:27.570607 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-258daa59-dfcf-4a00-a639-43400c29baa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:27.683269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 696/1405] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:32:27.728159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9860e0d-2639-499b-bfee-ff830a0d3b13 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:32:27.729707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b9860e0d-2639-499b-bfee-ff830a0d3b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:27.897774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:27.900429 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9860e0d-2639-499b-bfee-ff830a0d3b13 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:32:28.028541 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16653089-1404-43d7-9513-98d08171bbdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:32:28.029134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16653089-1404-43d7-9513-98d08171bbdf service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:28.029415 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16653089-1404-43d7-9513-98d08171bbdf service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:28.029726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16653089-1404-43d7-9513-98d08171bbdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:28.031372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-16653089-1404-43d7-9513-98d08171bbdf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:28.131607 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16653089-1404-43d7-9513-98d08171bbdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:32:28.133641 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16653089-1404-43d7-9513-98d08171bbdf service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:32:28.153266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 710/1406] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:32:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:32:30.285002 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f08bb-a582-4777-a6fb-bef300c9a85d None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:30.285733 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f08bb-a582-4777-a6fb-bef300c9a85d None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:30.286032 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f08bb-a582-4777-a6fb-bef300c9a85d None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:30.287360 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 697/1407] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:32:30.292567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159d434-3142-4850-80a5-c4490b09a83a None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:30.293027 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159d434-3142-4850-80a5-c4490b09a83a None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:30.293123 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159d434-3142-4850-80a5-c4490b09a83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:30.298030 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:30.367555 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-a159d434-3142-4850-80a5-c4490b09a83a None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:30.369195 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a159d434-3142-4850-80a5-c4490b09a83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:30.480035 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 711/1408] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:32:30.535329 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f08bb-a582-4777-a6fb-bef300c9a85d None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:32:30.538700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-9c0f08bb-a582-4777-a6fb-bef300c9a85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:30.634154 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:30.700767 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f08bb-a582-4777-a6fb-bef300c9a85d None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:32:30.832005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da5a0d-a5d3-4d19-865f-de3a823bb2ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:32:30.832629 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da5a0d-a5d3-4d19-865f-de3a823bb2ac service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:30.832910 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da5a0d-a5d3-4d19-865f-de3a823bb2ac service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:30.833188 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da5a0d-a5d3-4d19-865f-de3a823bb2ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:30.834883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-09da5a0d-a5d3-4d19-865f-de3a823bb2ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:30.894254 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:30.940026 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09da5a0d-a5d3-4d19-865f-de3a823bb2ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:32:30.941693 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09da5a0d-a5d3-4d19-865f-de3a823bb2ac service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:32:30.963088 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 698/1409] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:32:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:32:40.478258 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=64 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:40.960383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=23 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:47.828747 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e28330-dfd9-4c70-a447-82efa1fbba67 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:47.829376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e28330-dfd9-4c70-a447-82efa1fbba67 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:47.829376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e28330-dfd9-4c70-a447-82efa1fbba67 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:47.830854 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 712/1410] 10.222.0.47 () {54 vars in 918 bytes} [Thu Aug 4 04:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:32:47.837348 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71c7a5-7932-49ae-8b76-249e2681c0ef None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:47.837796 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71c7a5-7932-49ae-8b76-249e2681c0ef None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:47.837903 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71c7a5-7932-49ae-8b76-249e2681c0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:47.917758 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-2a71c7a5-7932-49ae-8b76-249e2681c0ef None None] MFA Rules not processed for user `350f72f740f749f68175c1c4e8ae9d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:47.918953 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-2a71c7a5-7932-49ae-8b76-249e2681c0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:48.027220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 699/1411] 10.222.0.47 () {58 vars in 1015 bytes} [Thu Aug 4 04:32:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:32:48.042230 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007907b-7933-4f6f-8c7a-2ef1078a7e16 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:48.042547 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007907b-7933-4f6f-8c7a-2ef1078a7e16 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:48.042826 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007907b-7933-4f6f-8c7a-2ef1078a7e16 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:48.044205 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 713/1412] 10.222.0.47 () {54 vars in 918 bytes} [Thu Aug 4 04:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:32:48.049160 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d98326-8f25-4ae0-b38b-61a80009d07c None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:48.049441 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d98326-8f25-4ae0-b38b-61a80009d07c None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:48.049738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d98326-8f25-4ae0-b38b-61a80009d07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:48.123080 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:48.126788 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-32d98326-8f25-4ae0-b38b-61a80009d07c None None] MFA Rules not processed for user `caa2901451aa4554bdfcac1bddbc620a`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:48.128965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-32d98326-8f25-4ae0-b38b-61a80009d07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:48.245984 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 700/1413] 10.222.0.47 () {58 vars in 1015 bytes} [Thu Aug 4 04:32:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:32:50.745311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee7581-82d1-4da0-81e8-2db8f4c9166b None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:50.745869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee7581-82d1-4da0-81e8-2db8f4c9166b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:50.745991 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee7581-82d1-4da0-81e8-2db8f4c9166b None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:50.747270 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 714/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 04:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:32:50.752554 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a9be1-e001-47ec-9c48-ded630b3dda1 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:50.752962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a9be1-e001-47ec-9c48-ded630b3dda1 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:50.753099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a9be1-e001-47ec-9c48-ded630b3dda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:50.825688 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-f24a9be1-e001-47ec-9c48-ded630b3dda1 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:50.826882 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-f24a9be1-e001-47ec-9c48-ded630b3dda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:50.887911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:50.931079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 701/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 04:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:32:50.975137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee7581-82d1-4da0-81e8-2db8f4c9166b None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:32:50.976663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-cfee7581-82d1-4da0-81e8-2db8f4c9166b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:51.123546 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee7581-82d1-4da0-81e8-2db8f4c9166b None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:32:51.197726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:51.242370 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b71305-449a-4ce4-8adc-e034c3640cc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:32:51.242986 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b71305-449a-4ce4-8adc-e034c3640cc7 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:51.243265 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b71305-449a-4ce4-8adc-e034c3640cc7 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:51.243537 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b71305-449a-4ce4-8adc-e034c3640cc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:51.245149 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c0b71305-449a-4ce4-8adc-e034c3640cc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:51.333908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b71305-449a-4ce4-8adc-e034c3640cc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:32:51.335510 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b71305-449a-4ce4-8adc-e034c3640cc7 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:32:51.353373 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 715/1416] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:32:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:32:52.254537 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f42439-3010-4540-ac50-2a81358df643 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:52.255072 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f42439-3010-4540-ac50-2a81358df643 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:52.255072 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f42439-3010-4540-ac50-2a81358df643 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:52.256353 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 702/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 04:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:32:52.261297 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6e939-dc16-4dca-a7c9-cd528f05222b None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:52.261733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6e939-dc16-4dca-a7c9-cd528f05222b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:52.261875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6e939-dc16-4dca-a7c9-cd528f05222b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:52.339206 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b6e6e939-dc16-4dca-a7c9-cd528f05222b None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:52.340400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b6e6e939-dc16-4dca-a7c9-cd528f05222b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:52.367039 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:52.437149 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 716/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 04:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:32:52.593391 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f42439-3010-4540-ac50-2a81358df643 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:32:52.594956 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-71f42439-3010-4540-ac50-2a81358df643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:52.746065 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f42439-3010-4540-ac50-2a81358df643 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:32:52.786462 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:52.859762 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93671d1e-b0b8-4a41-9fff-6e5c95ccc6c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:32:52.860378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93671d1e-b0b8-4a41-9fff-6e5c95ccc6c3 service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:52.860659 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93671d1e-b0b8-4a41-9fff-6e5c95ccc6c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:52.860933 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93671d1e-b0b8-4a41-9fff-6e5c95ccc6c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:52.862631 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-93671d1e-b0b8-4a41-9fff-6e5c95ccc6c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:52.950059 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93671d1e-b0b8-4a41-9fff-6e5c95ccc6c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:32:52.951646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93671d1e-b0b8-4a41-9fff-6e5c95ccc6c3 service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:32:52.970219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 703/1419] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:32:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:32:53.865870 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79570b2e-0b5b-4d19-bc51-d73a91c97790 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:53.865870 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79570b2e-0b5b-4d19-bc51-d73a91c97790 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:53.866518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79570b2e-0b5b-4d19-bc51-d73a91c97790 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:53.867647 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 717/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 04:32:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:32:53.872911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2853d86-5b9c-4a2f-8891-3924fd6c8097 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:53.873359 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2853d86-5b9c-4a2f-8891-3924fd6c8097 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:53.873501 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2853d86-5b9c-4a2f-8891-3924fd6c8097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:53.936239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:53.961833 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-b2853d86-5b9c-4a2f-8891-3924fd6c8097 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:53.963546 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-b2853d86-5b9c-4a2f-8891-3924fd6c8097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:54.076910 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 704/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 04:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:32:54.236845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79570b2e-0b5b-4d19-bc51-d73a91c97790 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:32:54.238900 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-79570b2e-0b5b-4d19-bc51-d73a91c97790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:54.403506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79570b2e-0b5b-4d19-bc51-d73a91c97790 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:32:54.416587 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:54.546863 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b146fc-01b2-4730-8c7c-3d8da908fb35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:32:54.547491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b146fc-01b2-4730-8c7c-3d8da908fb35 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:54.547869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b146fc-01b2-4730-8c7c-3d8da908fb35 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:54.548141 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b146fc-01b2-4730-8c7c-3d8da908fb35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:54.549831 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-59b146fc-01b2-4730-8c7c-3d8da908fb35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:54.663597 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b146fc-01b2-4730-8c7c-3d8da908fb35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:32:54.665211 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b146fc-01b2-4730-8c7c-3d8da908fb35 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:32:54.685490 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 718/1422] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:32:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:32:55.555876 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6490f52-c65f-40db-bea2-7f5ada5d756c None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:55.556424 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6490f52-c65f-40db-bea2-7f5ada5d756c None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:55.556424 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6490f52-c65f-40db-bea2-7f5ada5d756c None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:55.557806 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 705/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 04:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:32:55.562812 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f865da8-663b-4a4a-870d-46a3ad886d1d None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:55.563234 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f865da8-663b-4a4a-870d-46a3ad886d1d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:55.563373 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f865da8-663b-4a4a-870d-46a3ad886d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:55.583500 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:55.639549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-9f865da8-663b-4a4a-870d-46a3ad886d1d None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:55.640754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9f865da8-663b-4a4a-870d-46a3ad886d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:55.741479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 719/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 04:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:32:55.899435 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6490f52-c65f-40db-bea2-7f5ada5d756c None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:32:55.901024 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d6490f52-c65f-40db-bea2-7f5ada5d756c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:56.012415 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:56.049603 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6490f52-c65f-40db-bea2-7f5ada5d756c None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:32:56.163191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad934b38-52cc-4fbc-826d-204fafcb843c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:32:56.163727 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad934b38-52cc-4fbc-826d-204fafcb843c service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:56.164010 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad934b38-52cc-4fbc-826d-204fafcb843c service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:56.164287 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad934b38-52cc-4fbc-826d-204fafcb843c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:56.165952 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-ad934b38-52cc-4fbc-826d-204fafcb843c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:56.242728 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:56.257084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad934b38-52cc-4fbc-826d-204fafcb843c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:32:56.258691 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad934b38-52cc-4fbc-826d-204fafcb843c service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:32:56.277124 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 706/1425] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:32:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:32:57.142616 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d0ea1-aab0-48c3-8909-433c89960ff3 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:57.143187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d0ea1-aab0-48c3-8909-433c89960ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:57.143187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d0ea1-aab0-48c3-8909-433c89960ff3 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:57.144748 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 720/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 04:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:32:57.149801 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c31620-d371-47f6-bb6e-5f1ff2396377 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:57.150231 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c31620-d371-47f6-bb6e-5f1ff2396377 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:57.150345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c31620-d371-47f6-bb6e-5f1ff2396377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:57.230737 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-60c31620-d371-47f6-bb6e-5f1ff2396377 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:57.231943 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-60c31620-d371-47f6-bb6e-5f1ff2396377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:57.328434 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 707/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 04:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:32:57.482619 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d0ea1-aab0-48c3-8909-433c89960ff3 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:32:57.484055 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-be0d0ea1-aab0-48c3-8909-433c89960ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:57.569865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:57.633134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d0ea1-aab0-48c3-8909-433c89960ff3 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:32:57.745192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e97c94-8225-415e-a9af-715ab7c7ab49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:32:57.745752 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e97c94-8225-415e-a9af-715ab7c7ab49 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:57.746028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e97c94-8225-415e-a9af-715ab7c7ab49 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:57.746294 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e97c94-8225-415e-a9af-715ab7c7ab49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:57.747891 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d2e97c94-8225-415e-a9af-715ab7c7ab49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:57.795210 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:57.836195 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e97c94-8225-415e-a9af-715ab7c7ab49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:32:57.837818 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e97c94-8225-415e-a9af-715ab7c7ab49 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:32:57.856224 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 721/1428] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:32:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:32:58.688746 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf637f86-1ced-4315-b533-d4d386a9dbbb None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:58.689360 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf637f86-1ced-4315-b533-d4d386a9dbbb None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:58.689360 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf637f86-1ced-4315-b533-d4d386a9dbbb None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:58.691431 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 708/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 04:32:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:32:58.696650 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b02c3-bfca-48ea-b9e3-684a37c0e5b1 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:58.697142 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b02c3-bfca-48ea-b9e3-684a37c0e5b1 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:58.697262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b02c3-bfca-48ea-b9e3-684a37c0e5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:58.772590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7b6b02c3-bfca-48ea-b9e3-684a37c0e5b1 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:32:58.773729 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7b6b02c3-bfca-48ea-b9e3-684a37c0e5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:58.870664 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 722/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 04:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:32:59.025180 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf637f86-1ced-4315-b533-d4d386a9dbbb None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:32:59.026649 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-bf637f86-1ced-4315-b533-d4d386a9dbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:59.076828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:59.177465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf637f86-1ced-4315-b533-d4d386a9dbbb None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:32:59.292877 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7ed65-9eb6-4669-807b-977681d241ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:32:59.293487 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7ed65-9eb6-4669-807b-977681d241ec service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:32:59.293808 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7ed65-9eb6-4669-807b-977681d241ec service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:32:59.294061 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7ed65-9eb6-4669-807b-977681d241ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:32:59.295927 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-4ea7ed65-9eb6-4669-807b-977681d241ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:32:59.302875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:32:59.387543 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea7ed65-9eb6-4669-807b-977681d241ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:32:59.389110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea7ed65-9eb6-4669-807b-977681d241ec service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:32:59.407005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 709/1431] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:32:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:33:00.220558 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd5aa4-da8b-44cb-9a0a-b77172855798 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:00.221059 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd5aa4-da8b-44cb-9a0a-b77172855798 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:00.221176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd5aa4-da8b-44cb-9a0a-b77172855798 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:00.222595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 723/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 04:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:33:00.227758 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57748d59-1bbf-4066-b591-8821fa0294ed None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:00.228200 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57748d59-1bbf-4066-b591-8821fa0294ed None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:00.228476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57748d59-1bbf-4066-b591-8821fa0294ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:00.305370 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-57748d59-1bbf-4066-b591-8821fa0294ed None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:33:00.306727 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-57748d59-1bbf-4066-b591-8821fa0294ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:00.373376 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:00.402876 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 710/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 04:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:33:00.554379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd5aa4-da8b-44cb-9a0a-b77172855798 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:33:00.555918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-89dd5aa4-da8b-44cb-9a0a-b77172855798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:00.574167 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:00.703284 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd5aa4-da8b-44cb-9a0a-b77172855798 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:33:00.790681 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:00.817533 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d0745-6a30-40a6-9435-3ec6f2d0452b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:33:00.818182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d0745-6a30-40a6-9435-3ec6f2d0452b service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:00.818465 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d0745-6a30-40a6-9435-3ec6f2d0452b service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:00.818741 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d0745-6a30-40a6-9435-3ec6f2d0452b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:00.820365 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a25d0745-6a30-40a6-9435-3ec6f2d0452b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:00.910013 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25d0745-6a30-40a6-9435-3ec6f2d0452b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:33:00.911622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25d0745-6a30-40a6-9435-3ec6f2d0452b service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:33:00.929644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 724/1434] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:33:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:33:01.765875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a85c5-62b1-4c0a-be7e-9be6b68da389 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:01.766404 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a85c5-62b1-4c0a-be7e-9be6b68da389 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:01.766404 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a85c5-62b1-4c0a-be7e-9be6b68da389 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:01.767695 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 711/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 04:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:33:01.772770 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa392b80-a413-4356-b25c-25f255f71295 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:01.773208 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa392b80-a413-4356-b25c-25f255f71295 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:01.773316 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa392b80-a413-4356-b25c-25f255f71295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:01.848499 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-aa392b80-a413-4356-b25c-25f255f71295 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:33:01.849809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-aa392b80-a413-4356-b25c-25f255f71295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:01.891319 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:01.949150 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 725/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 04:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:33:02.108602 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a85c5-62b1-4c0a-be7e-9be6b68da389 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:33:02.110192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-7b5a85c5-62b1-4c0a-be7e-9be6b68da389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:02.257368 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a85c5-62b1-4c0a-be7e-9be6b68da389 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:33:02.312851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:02.373393 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f4a0b-d62b-4168-8ea3-165cfe1f127c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:33:02.373962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f4a0b-d62b-4168-8ea3-165cfe1f127c service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:02.374242 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f4a0b-d62b-4168-8ea3-165cfe1f127c service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:02.374514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f4a0b-d62b-4168-8ea3-165cfe1f127c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:02.376128 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-436f4a0b-d62b-4168-8ea3-165cfe1f127c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:02.464602 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436f4a0b-d62b-4168-8ea3-165cfe1f127c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:33:02.466219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436f4a0b-d62b-4168-8ea3-165cfe1f127c service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:33:02.484430 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 712/1437] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:33:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:33:03.302242 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97bbb64-78fb-4a97-b269-2cd86c2d9b00 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:03.302798 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97bbb64-78fb-4a97-b269-2cd86c2d9b00 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:03.302798 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97bbb64-78fb-4a97-b269-2cd86c2d9b00 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:03.304155 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 726/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 04:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:33:03.309454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b3205-a342-4c06-81e0-088c75344c70 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:03.309923 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b3205-a342-4c06-81e0-088c75344c70 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:03.310028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b3205-a342-4c06-81e0-088c75344c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:03.386554 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-202b3205-a342-4c06-81e0-088c75344c70 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:33:03.387756 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-202b3205-a342-4c06-81e0-088c75344c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:03.389767 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:03.485830 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 713/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 04:33:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:33:03.640421 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97bbb64-78fb-4a97-b269-2cd86c2d9b00 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:33:03.641960 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f97bbb64-78fb-4a97-b269-2cd86c2d9b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:03.789325 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97bbb64-78fb-4a97-b269-2cd86c2d9b00 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:33:03.817180 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:03.907444 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef682c-0f67-44c8-becd-992deddd96e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:33:03.908083 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef682c-0f67-44c8-becd-992deddd96e0 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:03.908362 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef682c-0f67-44c8-becd-992deddd96e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:03.908651 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef682c-0f67-44c8-becd-992deddd96e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:03.910885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7bef682c-0f67-44c8-becd-992deddd96e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:03.998707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef682c-0f67-44c8-becd-992deddd96e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:33:04.000301 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef682c-0f67-44c8-becd-992deddd96e0 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:33:04.018171 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 727/1440] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:33:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:33:05.984674 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523127a-4f8f-468d-9c1b-28a4b052fb8a None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:05.985126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523127a-4f8f-468d-9c1b-28a4b052fb8a None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:05.985225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523127a-4f8f-468d-9c1b-28a4b052fb8a None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:05.986618 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 714/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 04:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:33:05.993615 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de185daf-7404-4b74-b7da-dbf9a6b03a87 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:05.994116 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de185daf-7404-4b74-b7da-dbf9a6b03a87 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:05.994239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de185daf-7404-4b74-b7da-dbf9a6b03a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:06.041144 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:06.067602 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-de185daf-7404-4b74-b7da-dbf9a6b03a87 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:33:06.068996 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-de185daf-7404-4b74-b7da-dbf9a6b03a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:06.167488 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 728/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 04:33:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:33:06.208316 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d523127a-4f8f-468d-9c1b-28a4b052fb8a None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:33:06.209922 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d523127a-4f8f-468d-9c1b-28a4b052fb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:06.313598 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d523127a-4f8f-468d-9c1b-28a4b052fb8a None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:33:06.346092 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:06.388343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f6c11-d0d2-49ec-a5a1-11265c5b9ec7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:33:06.388976 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f6c11-d0d2-49ec-a5a1-11265c5b9ec7 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:06.389250 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f6c11-d0d2-49ec-a5a1-11265c5b9ec7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:06.389518 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f6c11-d0d2-49ec-a5a1-11265c5b9ec7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:06.391153 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-e00f6c11-d0d2-49ec-a5a1-11265c5b9ec7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:06.501228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f6c11-d0d2-49ec-a5a1-11265c5b9ec7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:33:06.502603 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f6c11-d0d2-49ec-a5a1-11265c5b9ec7 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:33:06.522364 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 715/1443] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:33:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:33:08.669472 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768509a-31ed-4695-b39a-2fab99addcd9 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:08.670348 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768509a-31ed-4695-b39a-2fab99addcd9 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:08.670348 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768509a-31ed-4695-b39a-2fab99addcd9 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:08.671709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 729/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 04:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:33:08.677340 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c54605-c068-439b-abdf-7bdd15841fb3 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:08.677825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c54605-c068-439b-abdf-7bdd15841fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:08.677938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c54605-c068-439b-abdf-7bdd15841fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:08.745860 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-82c54605-c068-439b-abdf-7bdd15841fb3 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:33:08.747100 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-82c54605-c068-439b-abdf-7bdd15841fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:08.760015 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:08.845143 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 716/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 04:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:33:08.883140 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768509a-31ed-4695-b39a-2fab99addcd9 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:33:08.884702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-5768509a-31ed-4695-b39a-2fab99addcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:08.996001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768509a-31ed-4695-b39a-2fab99addcd9 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:33:09.008632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:09.070051 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1340f9a-b46b-4bf8-8a89-5c59c30cf22d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:33:09.070663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1340f9a-b46b-4bf8-8a89-5c59c30cf22d service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:09.070939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1340f9a-b46b-4bf8-8a89-5c59c30cf22d service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:09.071204 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1340f9a-b46b-4bf8-8a89-5c59c30cf22d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:09.072803 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d1340f9a-b46b-4bf8-8a89-5c59c30cf22d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:09.184025 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1340f9a-b46b-4bf8-8a89-5c59c30cf22d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:33:09.185468 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1340f9a-b46b-4bf8-8a89-5c59c30cf22d service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:33:09.205688 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 730/1446] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:33:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:33:11.546099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc2614-ff09-4b1d-af63-5eafa936eabd None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:11.546632 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc2614-ff09-4b1d-af63-5eafa936eabd None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:11.546632 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc2614-ff09-4b1d-af63-5eafa936eabd None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:11.548006 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 717/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 04:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:33:11.554409 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4b4aa-2ce6-4832-9b41-37276faef417 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:11.554827 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4b4aa-2ce6-4832-9b41-37276faef417 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:11.554953 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4b4aa-2ce6-4832-9b41-37276faef417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:11.602979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:11.621474 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-64e4b4aa-2ce6-4832-9b41-37276faef417 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:33:11.622726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-64e4b4aa-2ce6-4832-9b41-37276faef417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:11.717507 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 731/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 04:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:33:11.757333 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc2614-ff09-4b1d-af63-5eafa936eabd None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:33:11.758845 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-b0bc2614-ff09-4b1d-af63-5eafa936eabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:11.861751 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc2614-ff09-4b1d-af63-5eafa936eabd None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:33:11.901524 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:11.937172 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b514f55-597b-40ba-b335-b95aac39b1d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:33:11.937841 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b514f55-597b-40ba-b335-b95aac39b1d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:11.938125 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b514f55-597b-40ba-b335-b95aac39b1d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:11.938398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b514f55-597b-40ba-b335-b95aac39b1d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:11.940020 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-1b514f55-597b-40ba-b335-b95aac39b1d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:12.052744 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b514f55-597b-40ba-b335-b95aac39b1d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:33:12.054215 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b514f55-597b-40ba-b335-b95aac39b1d3 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:33:12.072095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 718/1449] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:33:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:33:14.214759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50b12d-b8cd-4898-8880-88c07f0111ee None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:14.215290 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50b12d-b8cd-4898-8880-88c07f0111ee None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:14.215290 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50b12d-b8cd-4898-8880-88c07f0111ee None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:14.216655 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 732/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 04:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:33:14.222781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088aed00-a711-4602-8d53-ef380f583ca0 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:14.223690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088aed00-a711-4602-8d53-ef380f583ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:14.223971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088aed00-a711-4602-8d53-ef380f583ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:14.290632 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-088aed00-a711-4602-8d53-ef380f583ca0 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:33:14.291841 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-088aed00-a711-4602-8d53-ef380f583ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:14.312926 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:14.390878 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 719/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 04:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:33:14.431605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca50b12d-b8cd-4898-8880-88c07f0111ee None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:33:14.433139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ca50b12d-b8cd-4898-8880-88c07f0111ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:14.536702 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca50b12d-b8cd-4898-8880-88c07f0111ee None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:33:14.556474 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:14.610068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a969a2-8ac6-4740-9581-81b66d24c016 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:33:14.610716 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a969a2-8ac6-4740-9581-81b66d24c016 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:14.610997 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a969a2-8ac6-4740-9581-81b66d24c016 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:14.611272 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a969a2-8ac6-4740-9581-81b66d24c016 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:14.613059 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-04a969a2-8ac6-4740-9581-81b66d24c016 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:14.721281 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a969a2-8ac6-4740-9581-81b66d24c016 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:33:14.723328 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a969a2-8ac6-4740-9581-81b66d24c016 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:33:14.741004 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 733/1452] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:33:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:33:16.938815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fd6dc-afa5-41f0-a83b-2dc318ff95bb None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:16.939377 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fd6dc-afa5-41f0-a83b-2dc318ff95bb None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:16.939377 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fd6dc-afa5-41f0-a83b-2dc318ff95bb None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:16.940700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 720/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 04:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:33:16.946039 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207baf6-86db-4b05-ab10-56f559659f10 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:16.946479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207baf6-86db-4b05-ab10-56f559659f10 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:16.946583 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207baf6-86db-4b05-ab10-56f559659f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:17.011447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:17.014637 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b207baf6-86db-4b05-ab10-56f559659f10 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:33:17.016367 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b207baf6-86db-4b05-ab10-56f559659f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:17.115983 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 734/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 04:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:33:17.155350 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fd6dc-afa5-41f0-a83b-2dc318ff95bb None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:33:17.156877 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-cf7fd6dc-afa5-41f0-a83b-2dc318ff95bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:17.258116 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fd6dc-afa5-41f0-a83b-2dc318ff95bb None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:33:17.306797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:17.331465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26da51-1fca-4c19-a8c6-2006006bd31c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:33:17.332083 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26da51-1fca-4c19-a8c6-2006006bd31c service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:17.332362 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26da51-1fca-4c19-a8c6-2006006bd31c service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:17.332637 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26da51-1fca-4c19-a8c6-2006006bd31c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:17.334299 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-0c26da51-1fca-4c19-a8c6-2006006bd31c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:17.445767 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c26da51-1fca-4c19-a8c6-2006006bd31c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:33:17.447199 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c26da51-1fca-4c19-a8c6-2006006bd31c service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:33:17.465653 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 721/1455] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:33:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:33:19.580555 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebb8a7-c725-4dd0-ba2f-3ed39fc31b9d None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:19.581210 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebb8a7-c725-4dd0-ba2f-3ed39fc31b9d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:19.581210 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebb8a7-c725-4dd0-ba2f-3ed39fc31b9d None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:19.582600 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 735/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 04:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:33:19.587999 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e061a-e8ad-4f81-9643-433b04ea5bad None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:19.588402 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e061a-e8ad-4f81-9643-433b04ea5bad None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:19.588545 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e061a-e8ad-4f81-9643-433b04ea5bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:19.655928 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-a23e061a-e8ad-4f81-9643-433b04ea5bad None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:33:19.657133 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-a23e061a-e8ad-4f81-9643-433b04ea5bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:19.687239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:19.755814 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 722/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 04:33:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:33:19.795773 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebb8a7-c725-4dd0-ba2f-3ed39fc31b9d None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:33:19.797327 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-85ebb8a7-c725-4dd0-ba2f-3ed39fc31b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:19.900058 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebb8a7-c725-4dd0-ba2f-3ed39fc31b9d None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:33:19.929144 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:19.974985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef19d7-eff9-493a-972c-950ef1f82bb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:33:19.975657 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef19d7-eff9-493a-972c-950ef1f82bb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:19.975970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef19d7-eff9-493a-972c-950ef1f82bb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:19.976243 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef19d7-eff9-493a-972c-950ef1f82bb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:19.977918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-86ef19d7-eff9-493a-972c-950ef1f82bb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:20.089601 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ef19d7-eff9-493a-972c-950ef1f82bb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:33:20.091045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ef19d7-eff9-493a-972c-950ef1f82bb7 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:33:20.109521 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 736/1458] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:33:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:33:22.197626 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d87f4-4961-4216-9ee6-1114e15f4f1d None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:22.198157 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d87f4-4961-4216-9ee6-1114e15f4f1d None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:22.198157 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d87f4-4961-4216-9ee6-1114e15f4f1d None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:22.199482 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 723/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 04:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:33:22.204921 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123ff19-9f95-43a3-ae6e-2075ab865fce None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:22.205388 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123ff19-9f95-43a3-ae6e-2075ab865fce None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:22.205485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123ff19-9f95-43a3-ae6e-2075ab865fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:22.277822 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-d123ff19-9f95-43a3-ae6e-2075ab865fce None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:33:22.279039 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d123ff19-9f95-43a3-ae6e-2075ab865fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:22.288307 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:22.378131 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 737/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 04:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:33:22.415348 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d87f4-4961-4216-9ee6-1114e15f4f1d None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:33:22.416878 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c27d87f4-4961-4216-9ee6-1114e15f4f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:22.523715 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d87f4-4961-4216-9ee6-1114e15f4f1d None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:33:22.579271 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:22.598255 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d1b0b-2a00-43da-b60e-e3b9a9198589 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:33:22.598873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d1b0b-2a00-43da-b60e-e3b9a9198589 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:22.599157 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d1b0b-2a00-43da-b60e-e3b9a9198589 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:22.599431 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d1b0b-2a00-43da-b60e-e3b9a9198589 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:22.601056 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-b68d1b0b-2a00-43da-b60e-e3b9a9198589 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:22.709736 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d1b0b-2a00-43da-b60e-e3b9a9198589 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:33:22.711134 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d1b0b-2a00-43da-b60e-e3b9a9198589 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:33:22.729307 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 724/1461] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:33:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:33:24.793977 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749ea21-7b71-4eee-bda3-feed02fde868 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:24.794534 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749ea21-7b71-4eee-bda3-feed02fde868 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:24.794534 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749ea21-7b71-4eee-bda3-feed02fde868 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:24.796182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 738/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 04:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:33:24.801939 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d2cb8-d896-46b2-99af-54f20ec16761 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:24.802346 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d2cb8-d896-46b2-99af-54f20ec16761 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:24.802539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d2cb8-d896-46b2-99af-54f20ec16761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:24.869957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-e29d2cb8-d896-46b2-99af-54f20ec16761 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:33:24.871155 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-e29d2cb8-d896-46b2-99af-54f20ec16761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:24.911020 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:24.971313 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 725/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 04:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:33:25.010959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749ea21-7b71-4eee-bda3-feed02fde868 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:33:25.012519 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4749ea21-7b71-4eee-bda3-feed02fde868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:25.117629 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749ea21-7b71-4eee-bda3-feed02fde868 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:33:25.155793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:25.194203 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183dc945-a3b2-4db1-84f5-18fb9793f7bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:33:25.194813 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dc945-a3b2-4db1-84f5-18fb9793f7bc service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:25.195087 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dc945-a3b2-4db1-84f5-18fb9793f7bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:25.195403 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dc945-a3b2-4db1-84f5-18fb9793f7bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:25.197104 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-183dc945-a3b2-4db1-84f5-18fb9793f7bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:25.308841 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183dc945-a3b2-4db1-84f5-18fb9793f7bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:33:25.310299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183dc945-a3b2-4db1-84f5-18fb9793f7bc service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:33:25.328877 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 739/1464] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:33:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:33:27.405947 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ca0b1-4e5c-4507-a8e9-f184421fc7d9 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:27.406523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ca0b1-4e5c-4507-a8e9-f184421fc7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:27.406523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ca0b1-4e5c-4507-a8e9-f184421fc7d9 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:27.407967 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 726/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 04:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:33:27.414038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdbc45-c999-438b-a95f-beb6a43a83e3 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:27.414468 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdbc45-c999-438b-a95f-beb6a43a83e3 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:27.414580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdbc45-c999-438b-a95f-beb6a43a83e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:27.480797 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-24bdbc45-c999-438b-a95f-beb6a43a83e3 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:33:27.482173 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-24bdbc45-c999-438b-a95f-beb6a43a83e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:27.500517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:27.582518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 740/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 04:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:33:27.620516 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ca0b1-4e5c-4507-a8e9-f184421fc7d9 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:33:27.622080 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-f87ca0b1-4e5c-4507-a8e9-f184421fc7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:27.726387 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ca0b1-4e5c-4507-a8e9-f184421fc7d9 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:33:27.792461 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:27.801212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a97dc-2d7e-46a4-822a-156c64a6321f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:33:27.801962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a97dc-2d7e-46a4-822a-156c64a6321f service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:27.802240 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a97dc-2d7e-46a4-822a-156c64a6321f service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:27.802511 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a97dc-2d7e-46a4-822a-156c64a6321f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:27.804118 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-271a97dc-2d7e-46a4-822a-156c64a6321f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:27.914049 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a97dc-2d7e-46a4-822a-156c64a6321f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:33:27.915486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a97dc-2d7e-46a4-822a-156c64a6321f service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:33:27.933883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 727/1467] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:33:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:33:29.982210 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d99c9-fc2e-4e52-a223-fde4e9f70e09 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:29.982799 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d99c9-fc2e-4e52-a223-fde4e9f70e09 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:29.982799 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d99c9-fc2e-4e52-a223-fde4e9f70e09 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:29.984126 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 741/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 04:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:33:29.990104 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332cad6b-4b44-4a69-8a7b-3fdbecbe60a5 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:29.990521 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332cad6b-4b44-4a69-8a7b-3fdbecbe60a5 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:29.990658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332cad6b-4b44-4a69-8a7b-3fdbecbe60a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:30.056449 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-332cad6b-4b44-4a69-8a7b-3fdbecbe60a5 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:33:30.057672 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-332cad6b-4b44-4a69-8a7b-3fdbecbe60a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:30.106609 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:30.155079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 728/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 04:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:33:30.193401 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d99c9-fc2e-4e52-a223-fde4e9f70e09 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:33:30.195024 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ad1d99c9-fc2e-4e52-a223-fde4e9f70e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:30.302130 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d99c9-fc2e-4e52-a223-fde4e9f70e09 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:33:30.349697 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:30.376140 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bafdd8-9c33-4a91-8120-350a752991cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:33:30.376757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bafdd8-9c33-4a91-8120-350a752991cf service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:30.377033 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bafdd8-9c33-4a91-8120-350a752991cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:30.377411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bafdd8-9c33-4a91-8120-350a752991cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:30.379862 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-87bafdd8-9c33-4a91-8120-350a752991cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:30.489608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bafdd8-9c33-4a91-8120-350a752991cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:33:30.490963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bafdd8-9c33-4a91-8120-350a752991cf service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:33:30.510311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 742/1470] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:33:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:33:40.153265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=22 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:40.507774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=66 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:50.447537 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf2d024-ab41-4114-aaef-9b726f951542 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:50.448091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf2d024-ab41-4114-aaef-9b726f951542 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:50.448091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf2d024-ab41-4114-aaef-9b726f951542 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:50.449502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 729/1471] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:33:50.454572 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffcbfe-1960-43fa-a2fa-a5db31263b46 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:50.455520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffcbfe-1960-43fa-a2fa-a5db31263b46 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:50.455660 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffcbfe-1960-43fa-a2fa-a5db31263b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:50.524998 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-02ffcbfe-1960-43fa-a2fa-a5db31263b46 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:33:50.526247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-02ffcbfe-1960-43fa-a2fa-a5db31263b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:50.621114 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 743/1472] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:33:50.706875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e771f-b0f6-44fa-b598-51d1a6f212ff None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:50.707162 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e771f-b0f6-44fa-b598-51d1a6f212ff None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:50.707442 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e771f-b0f6-44fa-b598-51d1a6f212ff None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:50.708782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 730/1473] 10.222.0.47 () {54 vars in 966 bytes} [Thu Aug 4 04:33:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:33:50.714055 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bda5e-09f8-401b-8aa4-7b7839f6570d None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:50.714344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bda5e-09f8-401b-8aa4-7b7839f6570d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:50.714618 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bda5e-09f8-401b-8aa4-7b7839f6570d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:50.791164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-174bda5e-09f8-401b-8aa4-7b7839f6570d None None] MFA Rules not processed for user `b77a955866c54a43a7aaa271fefdec57`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:33:50.792360 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-174bda5e-09f8-401b-8aa4-7b7839f6570d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:50.812558 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:50.867100 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 744/1474] 10.222.0.47 () {58 vars in 1063 bytes} [Thu Aug 4 04:33:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:33:50.874635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e771f-b0f6-44fa-b598-51d1a6f212ff None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:33:50.876342 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-8c5e771f-b0f6-44fa-b598-51d1a6f212ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:50.980080 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e771f-b0f6-44fa-b598-51d1a6f212ff None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:33:51.052936 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f54a70-5866-4d12-8e97-b41dc6229501 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b77a955866c54a43a7aaa271fefdec57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:33:51.053574 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f54a70-5866-4d12-8e97-b41dc6229501 service glance] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:51.053856 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f54a70-5866-4d12-8e97-b41dc6229501 service glance] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:51.054126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f54a70-5866-4d12-8e97-b41dc6229501 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:51.055736 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-f8f54a70-5866-4d12-8e97-b41dc6229501 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:51.109312 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:51.165845 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f54a70-5866-4d12-8e97-b41dc6229501 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:33:51.167263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f54a70-5866-4d12-8e97-b41dc6229501 service glance] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:33:51.186228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 731/1475] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:33:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:33:53.278036 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11e9bc-48c0-41f1-ba6b-bedd4dc60dae None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:53.278582 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11e9bc-48c0-41f1-ba6b-bedd4dc60dae None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:53.278582 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11e9bc-48c0-41f1-ba6b-bedd4dc60dae None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:53.279949 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 745/1476] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:33:53.285300 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8c1f2-ac4c-4ff5-8d51-87410a4c1bfd None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:53.285760 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8c1f2-ac4c-4ff5-8d51-87410a4c1bfd None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:53.285871 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8c1f2-ac4c-4ff5-8d51-87410a4c1bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:53.352194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-a5c8c1f2-ac4c-4ff5-8d51-87410a4c1bfd None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:33:53.353880 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-a5c8c1f2-ac4c-4ff5-8d51-87410a4c1bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:53.461941 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 732/1477] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:33:53.486954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11e9bc-48c0-41f1-ba6b-bedd4dc60dae None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:33:53.488476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6e11e9bc-48c0-41f1-ba6b-bedd4dc60dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:53.614906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11e9bc-48c0-41f1-ba6b-bedd4dc60dae None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:33:53.664645 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:53.713979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191be3c-5c13-4166-b5ec-37d6858f8bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:33:53.714620 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191be3c-5c13-4166-b5ec-37d6858f8bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:53.714903 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191be3c-5c13-4166-b5ec-37d6858f8bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:53.715172 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191be3c-5c13-4166-b5ec-37d6858f8bd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:53.715862 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191be3c-5c13-4166-b5ec-37d6858f8bd8 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:33:53.717680 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: WARNING py.warnings [None req-1191be3c-5c13-4166-b5ec-37d6858f8bd8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 04:33:53.717680 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: warnings.warn(msg) Aug 04 04:33:53.717680 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: Aug 04 04:33:53.718824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191be3c-5c13-4166-b5ec-37d6858f8bd8 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:33:53.738729 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 746/1478] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Aug 4 04:33:53 2022] GET /identity/v3/projects => generated 2553 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:33:55.854858 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9e876-3267-4970-b34f-c85a5ef23b9b None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:55.855496 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9e876-3267-4970-b34f-c85a5ef23b9b None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:55.855496 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9e876-3267-4970-b34f-c85a5ef23b9b None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:55.856858 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 733/1479] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:33:55.862152 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb8dab-2392-40e3-a8b9-8c208ee85fb5 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:55.862562 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb8dab-2392-40e3-a8b9-8c208ee85fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:55.862710 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb8dab-2392-40e3-a8b9-8c208ee85fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:55.933347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-48bb8dab-2392-40e3-a8b9-8c208ee85fb5 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:33:55.934594 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-48bb8dab-2392-40e3-a8b9-8c208ee85fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:56.032566 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 747/1480] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:33:56.077038 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9e876-3267-4970-b34f-c85a5ef23b9b None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:33:56.078715 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-77a9e876-3267-4970-b34f-c85a5ef23b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:56.089444 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:56.225467 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9e876-3267-4970-b34f-c85a5ef23b9b None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:33:56.306020 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:56.339633 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322497a9-8a46-403f-a041-a012f32bc16e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:33:56.340279 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322497a9-8a46-403f-a041-a012f32bc16e service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:56.340585 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322497a9-8a46-403f-a041-a012f32bc16e service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:56.340836 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322497a9-8a46-403f-a041-a012f32bc16e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:56.342526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-322497a9-8a46-403f-a041-a012f32bc16e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:56.432887 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322497a9-8a46-403f-a041-a012f32bc16e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:33:56.434513 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322497a9-8a46-403f-a041-a012f32bc16e service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:33:56.452970 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 734/1481] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:33:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:33:58.690139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd262f-9921-4d0b-afae-9679a3b069ac None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:58.690715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd262f-9921-4d0b-afae-9679a3b069ac None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:58.690715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd262f-9921-4d0b-afae-9679a3b069ac None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:58.692061 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 748/1482] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:33:58.697303 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b8b60-423c-4271-aa5a-fcc56de56a98 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:58.697759 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b8b60-423c-4271-aa5a-fcc56de56a98 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:58.697883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b8b60-423c-4271-aa5a-fcc56de56a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:58.764062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-c96b8b60-423c-4271-aa5a-fcc56de56a98 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:33:58.765173 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c96b8b60-423c-4271-aa5a-fcc56de56a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:58.810850 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:58.861897 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 735/1483] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:33:58.899374 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd262f-9921-4d0b-afae-9679a3b069ac None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:33:58.901028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-26bd262f-9921-4d0b-afae-9679a3b069ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:58.924601 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:59.125172 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd262f-9921-4d0b-afae-9679a3b069ac None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:33:59.198343 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37712717-98db-49b6-a0da-19dacd8c2fe7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:33:59.198955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37712717-98db-49b6-a0da-19dacd8c2fe7 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:33:59.199235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37712717-98db-49b6-a0da-19dacd8c2fe7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:33:59.199504 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37712717-98db-49b6-a0da-19dacd8c2fe7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:33:59.201114 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-37712717-98db-49b6-a0da-19dacd8c2fe7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:33:59.274156 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:33:59.310246 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37712717-98db-49b6-a0da-19dacd8c2fe7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:33:59.311681 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37712717-98db-49b6-a0da-19dacd8c2fe7 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:33:59.329442 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 749/1484] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:33:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:34:01.476722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99edcd-8565-466d-9224-7f1b97e45266 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:34:01.477323 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99edcd-8565-466d-9224-7f1b97e45266 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:34:01.477323 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99edcd-8565-466d-9224-7f1b97e45266 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:34:01.478665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 736/1485] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:34:01.484023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2ae96-d1bd-4875-946b-5a803bcec1e0 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:34:01.484453 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2ae96-d1bd-4875-946b-5a803bcec1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:34:01.484557 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2ae96-d1bd-4875-946b-5a803bcec1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:34:01.550614 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-d9b2ae96-d1bd-4875-946b-5a803bcec1e0 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:34:01.551849 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d9b2ae96-d1bd-4875-946b-5a803bcec1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:34:01.646774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 750/1486] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:34:01.686001 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99edcd-8565-466d-9224-7f1b97e45266 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:34:01.687604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-fe99edcd-8565-466d-9224-7f1b97e45266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:34:01.792094 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99edcd-8565-466d-9224-7f1b97e45266 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:34:01.869680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:34:01.871407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d3344-5275-487f-bca3-10c205684dd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:34:01.872023 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d3344-5275-487f-bca3-10c205684dd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:34:01.872573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d3344-5275-487f-bca3-10c205684dd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:34:01.873035 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d3344-5275-487f-bca3-10c205684dd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:34:01.874917 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-720d3344-5275-487f-bca3-10c205684dd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:34:01.982970 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d3344-5275-487f-bca3-10c205684dd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:34:01.984434 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d3344-5275-487f-bca3-10c205684dd8 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:34:02.002806 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 737/1487] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:34:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:34:04.129502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794f23f-d6b0-46ff-9b40-16e988946749 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:34:04.130109 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794f23f-d6b0-46ff-9b40-16e988946749 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:34:04.130109 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794f23f-d6b0-46ff-9b40-16e988946749 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:34:04.131441 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 751/1488] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 04:34:04.136615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2da8f4-6fae-47e8-a37e-384feb89ea76 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:34:04.137010 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2da8f4-6fae-47e8-a37e-384feb89ea76 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:34:04.137159 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2da8f4-6fae-47e8-a37e-384feb89ea76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:34:04.203574 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-4a2da8f4-6fae-47e8-a37e-384feb89ea76 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:34:04.204766 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-4a2da8f4-6fae-47e8-a37e-384feb89ea76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:34:04.259086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:34:04.304206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 738/1489] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:34:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:34:04.344720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5794f23f-d6b0-46ff-9b40-16e988946749 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:34:04.346981 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-5794f23f-d6b0-46ff-9b40-16e988946749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:34:04.383699 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:34:04.452321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5794f23f-d6b0-46ff-9b40-16e988946749 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:34:04.525868 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b570099-7f90-455f-9779-6e19dadaec4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:34:04.526482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b570099-7f90-455f-9779-6e19dadaec4a service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:34:04.526760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b570099-7f90-455f-9779-6e19dadaec4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:34:04.527046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b570099-7f90-455f-9779-6e19dadaec4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:34:04.528657 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8b570099-7f90-455f-9779-6e19dadaec4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:34:04.610263 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:34:04.639754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b570099-7f90-455f-9779-6e19dadaec4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:34:04.641182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b570099-7f90-455f-9779-6e19dadaec4a service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:34:04.659351 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 752/1490] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:34:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:34:06.891595 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d3b70-ceb5-4ad5-b7b4-e736e28a5029 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:34:06.892246 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d3b70-ceb5-4ad5-b7b4-e736e28a5029 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:34:06.892246 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d3b70-ceb5-4ad5-b7b4-e736e28a5029 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:34:06.893717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 739/1491] 10.222.0.47 () {54 vars in 927 bytes} [Thu Aug 4 04:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:34:06.898979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111a2dc-7d19-4ba1-b37e-79b30b917c8e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:34:06.899417 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111a2dc-7d19-4ba1-b37e-79b30b917c8e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:34:06.899523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111a2dc-7d19-4ba1-b37e-79b30b917c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:34:06.967591 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-0111a2dc-7d19-4ba1-b37e-79b30b917c8e None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:34:06.968794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0111a2dc-7d19-4ba1-b37e-79b30b917c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:34:07.065633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 753/1492] 10.222.0.47 () {58 vars in 1024 bytes} [Thu Aug 4 04:34:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:34:07.108014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d3b70-ceb5-4ad5-b7b4-e736e28a5029 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:34:07.109606 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-420d3b70-ceb5-4ad5-b7b4-e736e28a5029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:34:07.217025 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d3b70-ceb5-4ad5-b7b4-e736e28a5029 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:34:07.295496 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8434a-7e66-4dfc-9c6a-14e1831289d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:34:07.296382 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8434a-7e66-4dfc-9c6a-14e1831289d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:34:07.296660 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8434a-7e66-4dfc-9c6a-14e1831289d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:34:07.296932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8434a-7e66-4dfc-9c6a-14e1831289d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:34:07.299062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c1c8434a-7e66-4dfc-9c6a-14e1831289d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:34:07.308488 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:34:07.409860 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c8434a-7e66-4dfc-9c6a-14e1831289d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:34:07.411292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c8434a-7e66-4dfc-9c6a-14e1831289d3 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:34:07.431268 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 740/1493] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:34:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:34:17.064068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=82 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:34:17.428035 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=50 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:35:19.694107 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa259af-bb45-4195-a944-e5602f5cf2de None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:35:19.694707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa259af-bb45-4195-a944-e5602f5cf2de None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:35:19.694707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa259af-bb45-4195-a944-e5602f5cf2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:35:19.773073 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-eaa259af-bb45-4195-a944-e5602f5cf2de None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:35:19.774214 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-eaa259af-bb45-4195-a944-e5602f5cf2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:35:19.866854 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 754/1494] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:35:19.875730 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71f68f-9d83-47bc-9620-f5b5a77679c0 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:35:19.876278 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71f68f-9d83-47bc-9620-f5b5a77679c0 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:35:19.876278 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71f68f-9d83-47bc-9620-f5b5a77679c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:35:19.952699 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-4d71f68f-9d83-47bc-9620-f5b5a77679c0 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:35:19.954088 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-4d71f68f-9d83-47bc-9620-f5b5a77679c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:35:20.051291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 741/1495] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:35:20.055998 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa259af-bb45-4195-a944-e5602f5cf2de None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:35:20.057285 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-eaa259af-bb45-4195-a944-e5602f5cf2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:35:20.069211 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa259af-bb45-4195-a944-e5602f5cf2de None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:35:20.130306 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:35:20.164296 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3827b3-3301-442e-aee0-68a945ffadd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:35:20.164914 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3827b3-3301-442e-aee0-68a945ffadd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:35:20.165194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3827b3-3301-442e-aee0-68a945ffadd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:35:20.165466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3827b3-3301-442e-aee0-68a945ffadd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:35:20.166182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3827b3-3301-442e-aee0-68a945ffadd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:35:20.166485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3827b3-3301-442e-aee0-68a945ffadd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:35:20.167493 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: WARNING py.warnings [None req-2d3827b3-3301-442e-aee0-68a945ffadd5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 04:35:20.167493 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: warnings.warn(msg) Aug 04 04:35:20.167493 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: Aug 04 04:35:20.168354 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3827b3-3301-442e-aee0-68a945ffadd5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:35:20.180455 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 755/1496] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:35:20.188916 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71f68f-9d83-47bc-9620-f5b5a77679c0 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:35:20.198167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71f68f-9d83-47bc-9620-f5b5a77679c0 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:35:20.262246 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:35:20.297622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae59a38-850c-42f9-aee6-a264bc7118cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:35:20.298240 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae59a38-850c-42f9-aee6-a264bc7118cc admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:35:20.298522 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae59a38-850c-42f9-aee6-a264bc7118cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:35:20.298795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae59a38-850c-42f9-aee6-a264bc7118cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:35:20.299455 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae59a38-850c-42f9-aee6-a264bc7118cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:35:20.301189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: WARNING py.warnings [None req-eae59a38-850c-42f9-aee6-a264bc7118cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 04:35:20.301189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: warnings.warn(msg) Aug 04 04:35:20.301189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: Aug 04 04:35:20.302452 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae59a38-850c-42f9-aee6-a264bc7118cc admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:35:20.357876 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 742/1497] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:35:20 2022] POST /identity/v3/projects => generated 375 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:35:20.363041 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3827b3-3301-442e-aee0-68a945ffadd5 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:35:20.364608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2d3827b3-3301-442e-aee0-68a945ffadd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:35:20.397822 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3827b3-3301-442e-aee0-68a945ffadd5 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:35:20.496095 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa95ca0-0b16-4908-93b3-20baebac64d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:35:20.496704 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa95ca0-0b16-4908-93b3-20baebac64d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:35:20.496982 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa95ca0-0b16-4908-93b3-20baebac64d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:35:20.497250 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa95ca0-0b16-4908-93b3-20baebac64d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:35:20.497978 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa95ca0-0b16-4908-93b3-20baebac64d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:35:20.499055 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: WARNING py.warnings [None req-eaa95ca0-0b16-4908-93b3-20baebac64d8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 04:35:20.499055 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: warnings.warn(msg) Aug 04 04:35:20.499055 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: Aug 04 04:35:20.500399 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa95ca0-0b16-4908-93b3-20baebac64d8 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:35:20.546083 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 756/1498] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:35:20 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:35:20.551779 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae59a38-850c-42f9-aee6-a264bc7118cc admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:35:20.551981 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-eae59a38-850c-42f9-aee6-a264bc7118cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:35:20.614875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae59a38-850c-42f9-aee6-a264bc7118cc admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:35:20.696959 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:35:20.709216 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66844049-e4a9-45c0-b3c9-f1c31d92a095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:35:20.709870 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66844049-e4a9-45c0-b3c9-f1c31d92a095 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:35:20.710153 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66844049-e4a9-45c0-b3c9-f1c31d92a095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:35:20.710426 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66844049-e4a9-45c0-b3c9-f1c31d92a095 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:35:20.711123 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66844049-e4a9-45c0-b3c9-f1c31d92a095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:35:20.712129 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: WARNING py.warnings [None req-66844049-e4a9-45c0-b3c9-f1c31d92a095 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 04:35:20.712129 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: warnings.warn(msg) Aug 04 04:35:20.712129 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: Aug 04 04:35:20.713003 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66844049-e4a9-45c0-b3c9-f1c31d92a095 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:35:20.726107 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 743/1499] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:35:20.730396 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa95ca0-0b16-4908-93b3-20baebac64d8 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:35:20.731902 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-eaa95ca0-0b16-4908-93b3-20baebac64d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:35:20.743679 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa95ca0-0b16-4908-93b3-20baebac64d8 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:35:20.769236 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:35:20.842485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df0477-b8a0-4691-9629-08b1a093c076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:35:20.843125 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df0477-b8a0-4691-9629-08b1a093c076 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:35:20.843403 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df0477-b8a0-4691-9629-08b1a093c076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:35:20.843668 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df0477-b8a0-4691-9629-08b1a093c076 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:35:20.844357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df0477-b8a0-4691-9629-08b1a093c076 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:35:20.845389 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: WARNING py.warnings [None req-f5df0477-b8a0-4691-9629-08b1a093c076 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 04:35:20.845389 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: warnings.warn(msg) Aug 04 04:35:20.845389 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: Aug 04 04:35:20.846316 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df0477-b8a0-4691-9629-08b1a093c076 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:35:20.858563 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 757/1500] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:35:20.862936 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66844049-e4a9-45c0-b3c9-f1c31d92a095 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:35:20.864466 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-66844049-e4a9-45c0-b3c9-f1c31d92a095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:35:20.897767 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66844049-e4a9-45c0-b3c9-f1c31d92a095 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:35:20.991584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9740f4-ac20-4547-ac77-88c45fbb0529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:35:20.992242 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9740f4-ac20-4547-ac77-88c45fbb0529 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:35:20.992516 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9740f4-ac20-4547-ac77-88c45fbb0529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:35:20.992781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9740f4-ac20-4547-ac77-88c45fbb0529 admin admin] PATH_INFO: `/v3/projects/f2659e37c4f14e43a343aaadfab9eb82/users/b036cf5fc08f42638a0baabffce316af/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:35:21.014483 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9740f4-ac20-4547-ac77-88c45fbb0529 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2659e37c4f14e43a343aaadfab9eb82, user_id=b036cf5fc08f42638a0baabffce316af, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:35:21.016356 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: WARNING py.warnings [None req-6f9740f4-ac20-4547-ac77-88c45fbb0529 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 04:35:21.016356 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: warnings.warn(msg) Aug 04 04:35:21.016356 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: Aug 04 04:35:21.018282 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9740f4-ac20-4547-ac77-88c45fbb0529 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:35:21.041824 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 744/1501] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:35:20 2022] PUT /identity/v3/projects/f2659e37c4f14e43a343aaadfab9eb82/users/b036cf5fc08f42638a0baabffce316af/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:35:21.054828 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1003cc-c683-442b-a011-e9550ba6d839 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:35:21.055119 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1003cc-c683-442b-a011-e9550ba6d839 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:35:21.055395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1003cc-c683-442b-a011-e9550ba6d839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:35:21.113733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-6b1003cc-c683-442b-a011-e9550ba6d839 None None] MFA Rules not processed for user `b036cf5fc08f42638a0baabffce316af`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:35:21.114942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6b1003cc-c683-442b-a011-e9550ba6d839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:35:21.191889 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 758/1502] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:35:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:35:21.205647 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d32832-2204-4e4a-86a1-c82f64a9b9af None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:35:21.205953 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d32832-2204-4e4a-86a1-c82f64a9b9af None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:35:21.206224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d32832-2204-4e4a-86a1-c82f64a9b9af None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:35:21.207586 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 745/1503] 10.222.0.47 () {54 vars in 964 bytes} [Thu Aug 4 04:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:35:21.212616 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34feb25d-5ac7-469a-9016-f0e6566d9ab4 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:35:21.212906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34feb25d-5ac7-469a-9016-f0e6566d9ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:35:21.213182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34feb25d-5ac7-469a-9016-f0e6566d9ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:35:21.244103 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:35:21.288564 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-34feb25d-5ac7-469a-9016-f0e6566d9ab4 None None] MFA Rules not processed for user `8b1d67b28dca49ca9cb0ff28cad62aa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:35:21.289798 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-34feb25d-5ac7-469a-9016-f0e6566d9ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:35:21.364674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 759/1504] 10.222.0.47 () {58 vars in 1061 bytes} [Thu Aug 4 04:35:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:35:21.372538 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d32832-2204-4e4a-86a1-c82f64a9b9af None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:35:21.374169 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c8d32832-2204-4e4a-86a1-c82f64a9b9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:35:21.425531 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:35:21.479889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d32832-2204-4e4a-86a1-c82f64a9b9af None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:35:21.556524 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bc928-416e-44a4-abd7-ee5cee96bdaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:35:21.557140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bc928-416e-44a4-abd7-ee5cee96bdaf service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:35:21.557424 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bc928-416e-44a4-abd7-ee5cee96bdaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:35:21.557730 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bc928-416e-44a4-abd7-ee5cee96bdaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:35:21.559349 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-bb6bc928-416e-44a4-abd7-ee5cee96bdaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:35:21.659066 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6bc928-416e-44a4-abd7-ee5cee96bdaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:35:21.660492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6bc928-416e-44a4-abd7-ee5cee96bdaf service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:35:21.663986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:35:21.679220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 746/1505] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:35:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:35:21.724949 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7b8a8-436b-478a-9b54-d3764149e055 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:35:21.725242 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7b8a8-436b-478a-9b54-d3764149e055 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:35:21.725522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7b8a8-436b-478a-9b54-d3764149e055 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:35:21.726857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 760/1506] 10.222.0.47 () {52 vars in 830 bytes} [Thu Aug 4 04:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:35:21.730396 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bc928-416e-44a4-abd7-ee5cee96bdaf service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:35:21.731551 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-bb6bc928-416e-44a4-abd7-ee5cee96bdaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:35:21.828365 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bc928-416e-44a4-abd7-ee5cee96bdaf service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:35:21.920001 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e38dca-c16a-4fe7-8d8a-8b265671ecc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:35:21.920669 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e38dca-c16a-4fe7-8d8a-8b265671ecc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:35:21.920912 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e38dca-c16a-4fe7-8d8a-8b265671ecc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:35:21.921757 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e38dca-c16a-4fe7-8d8a-8b265671ecc8 admin admin] PATH_INFO: `/v3/users/b036cf5fc08f42638a0baabffce316af` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:35:21.933443 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e38dca-c16a-4fe7-8d8a-8b265671ecc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b036cf5fc08f42638a0baabffce316af)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:35:21.934506 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: WARNING py.warnings [None req-01e38dca-c16a-4fe7-8d8a-8b265671ecc8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 04:35:21.934506 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: warnings.warn(msg) Aug 04 04:35:21.934506 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: Aug 04 04:35:21.935696 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e38dca-c16a-4fe7-8d8a-8b265671ecc8 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:35:21.946002 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:35:22.009015 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-01e38dca-c16a-4fe7-8d8a-8b265671ecc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b036cf5fc08f42638a0baabffce316af'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:35:22.009446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-01e38dca-c16a-4fe7-8d8a-8b265671ecc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b036cf5fc08f42638a0baabffce316af'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:35:22.032362 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-01e38dca-c16a-4fe7-8d8a-8b265671ecc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b036cf5fc08f42638a0baabffce316af'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:35:22.041351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-01e38dca-c16a-4fe7-8d8a-8b265671ecc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b036cf5fc08f42638a0baabffce316af'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:35:22.041947 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-01e38dca-c16a-4fe7-8d8a-8b265671ecc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b036cf5fc08f42638a0baabffce316af'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:35:22.062212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 747/1507] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:35:21 2022] DELETE /identity/v3/users/b036cf5fc08f42638a0baabffce316af => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:35:22.067064 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01387b5-079d-41b4-be8e-dbb9329aca79 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:35:22.067545 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01387b5-079d-41b4-be8e-dbb9329aca79 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:35:22.067650 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01387b5-079d-41b4-be8e-dbb9329aca79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:35:22.123676 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-c01387b5-079d-41b4-be8e-dbb9329aca79 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:35:22.125640 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c01387b5-079d-41b4-be8e-dbb9329aca79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:35:22.200618 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:35:22.214027 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 761/1508] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:35:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:35:22.223721 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e38dca-c16a-4fe7-8d8a-8b265671ecc8 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:35:22.224983 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-01e38dca-c16a-4fe7-8d8a-8b265671ecc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:35:22.324425 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e38dca-c16a-4fe7-8d8a-8b265671ecc8 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:35:22.379813 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:35:22.400923 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e21645-1375-4a77-8f6e-6c50a895204c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:35:22.401567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e21645-1375-4a77-8f6e-6c50a895204c service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:35:22.401868 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e21645-1375-4a77-8f6e-6c50a895204c service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:35:22.402137 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e21645-1375-4a77-8f6e-6c50a895204c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:35:22.403768 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d2e21645-1375-4a77-8f6e-6c50a895204c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:35:22.511564 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e21645-1375-4a77-8f6e-6c50a895204c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:35:22.513029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e21645-1375-4a77-8f6e-6c50a895204c service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:35:22.532778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 748/1509] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:35:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:35:22.906712 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01387b5-079d-41b4-be8e-dbb9329aca79 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:35:22.908027 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c01387b5-079d-41b4-be8e-dbb9329aca79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:35:22.919631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01387b5-079d-41b4-be8e-dbb9329aca79 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:35:23.017821 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f2f6f-27ae-49d0-a6e2-7d9755d845cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:35:23.018458 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f2f6f-27ae-49d0-a6e2-7d9755d845cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:35:23.018737 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f2f6f-27ae-49d0-a6e2-7d9755d845cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:35:23.019008 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f2f6f-27ae-49d0-a6e2-7d9755d845cd admin admin] PATH_INFO: `/v3/projects/f2659e37c4f14e43a343aaadfab9eb82` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:35:23.028767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f2f6f-27ae-49d0-a6e2-7d9755d845cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2659e37c4f14e43a343aaadfab9eb82)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:35:23.029872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: WARNING py.warnings [None req-2b3f2f6f-27ae-49d0-a6e2-7d9755d845cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 04:35:23.029872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: warnings.warn(msg) Aug 04 04:35:23.029872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: Aug 04 04:35:23.032098 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f2f6f-27ae-49d0-a6e2-7d9755d845cd admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:35:23.098713 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2b3f2f6f-27ae-49d0-a6e2-7d9755d845cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2659e37c4f14e43a343aaadfab9eb82', 'request_id': 'req-2b3f2f6f-27ae-49d0-a6e2-7d9755d845cd'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:35:23.110565 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2b3f2f6f-27ae-49d0-a6e2-7d9755d845cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2659e37c4f14e43a343aaadfab9eb82', 'request_id': 'req-2b3f2f6f-27ae-49d0-a6e2-7d9755d845cd'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:35:23.117927 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2b3f2f6f-27ae-49d0-a6e2-7d9755d845cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2659e37c4f14e43a343aaadfab9eb82', 'request_id': 'req-2b3f2f6f-27ae-49d0-a6e2-7d9755d845cd'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:35:23.148021 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2b3f2f6f-27ae-49d0-a6e2-7d9755d845cd admin admin] The token cache is being invalidate because project f2659e37c4f14e43a343aaadfab9eb82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:35:23.148307 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2b3f2f6f-27ae-49d0-a6e2-7d9755d845cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:35:23.148739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2b3f2f6f-27ae-49d0-a6e2-7d9755d845cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:35:23.150179 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 762/1510] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:35:22 2022] DELETE /identity/v3/projects/f2659e37c4f14e43a343aaadfab9eb82 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:35:32.529718 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=62 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:35:33.146007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:36:50.685712 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bbe45-7c4b-4c33-8f09-09860d6f0825 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:36:50.686326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bbe45-7c4b-4c33-8f09-09860d6f0825 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:36:50.686326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bbe45-7c4b-4c33-8f09-09860d6f0825 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:36:50.687614 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 749/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 04:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:36:50.692534 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c598b-a9bb-4c04-b984-07cea7a4c847 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:36:50.693003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c598b-a9bb-4c04-b984-07cea7a4c847 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:36:50.693108 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c598b-a9bb-4c04-b984-07cea7a4c847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:36:50.775701 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-427c598b-a9bb-4c04-b984-07cea7a4c847 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:36:50.777421 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-427c598b-a9bb-4c04-b984-07cea7a4c847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:36:50.887285 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 763/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 04:36:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:36:51.044739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24bbe45-7c4b-4c33-8f09-09860d6f0825 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:36:51.047230 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c24bbe45-7c4b-4c33-8f09-09860d6f0825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:36:51.178856 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24bbe45-7c4b-4c33-8f09-09860d6f0825 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:36:51.185567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951b707-6a8c-4de9-9fdc-03a3fd34a647 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:36:51.186199 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951b707-6a8c-4de9-9fdc-03a3fd34a647 service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:36:51.186828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951b707-6a8c-4de9-9fdc-03a3fd34a647 service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:36:51.186828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951b707-6a8c-4de9-9fdc-03a3fd34a647 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:36:51.195661 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951b707-6a8c-4de9-9fdc-03a3fd34a647 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:36:51.197271 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951b707-6a8c-4de9-9fdc-03a3fd34a647 service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:36:51.208318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 750/1513] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:36:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:36:52.083216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669060b-ef28-401d-92df-cc8f08eaf8ca None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:36:52.083818 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669060b-ef28-401d-92df-cc8f08eaf8ca None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:36:52.083818 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669060b-ef28-401d-92df-cc8f08eaf8ca None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:36:52.085128 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 764/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 04:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:36:52.091998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54c15a-c8c0-4d83-8b3e-72e05e99aa7b None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:36:52.092547 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54c15a-c8c0-4d83-8b3e-72e05e99aa7b None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:36:52.092802 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54c15a-c8c0-4d83-8b3e-72e05e99aa7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:36:52.115029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-5e54c15a-c8c0-4d83-8b3e-72e05e99aa7b None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:36:52.116287 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5e54c15a-c8c0-4d83-8b3e-72e05e99aa7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:36:52.124962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 751/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 04:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:36:52.278039 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669060b-ef28-401d-92df-cc8f08eaf8ca None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:36:52.292950 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669060b-ef28-401d-92df-cc8f08eaf8ca None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:36:52.299585 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9472c6-8788-48ad-bbe7-cda5b7972916 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:36:52.300205 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9472c6-8788-48ad-bbe7-cda5b7972916 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:36:52.300483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9472c6-8788-48ad-bbe7-cda5b7972916 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:36:52.300756 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9472c6-8788-48ad-bbe7-cda5b7972916 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:36:52.310357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9472c6-8788-48ad-bbe7-cda5b7972916 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:36:52.311946 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9472c6-8788-48ad-bbe7-cda5b7972916 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:36:52.321700 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 765/1516] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:36:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:36:52.487175 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e172dd-d386-491a-a404-1d6806f2c63e None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:36:52.487571 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e172dd-d386-491a-a404-1d6806f2c63e None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:36:52.487858 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e172dd-d386-491a-a404-1d6806f2c63e None None] PATH_INFO: `/v3` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:36:52.489206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 752/1517] 10.222.0.65 () {54 vars in 925 bytes} [Thu Aug 4 04:36:52 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:36:52.495495 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6553280-4d2b-4e58-9ddf-99c4103529c1 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:36:52.496482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6553280-4d2b-4e58-9ddf-99c4103529c1 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:36:52.496763 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6553280-4d2b-4e58-9ddf-99c4103529c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:36:52.563350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-c6553280-4d2b-4e58-9ddf-99c4103529c1 None None] MFA Rules not processed for user `350f72f740f749f68175c1c4e8ae9d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:36:52.564529 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c6553280-4d2b-4e58-9ddf-99c4103529c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:36:52.583257 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:36:52.654867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 766/1518] 10.222.0.65 () {58 vars in 1015 bytes} [Thu Aug 4 04:36:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:36:56.474091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878ff00-3751-4549-b0eb-4f536103c4f1 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:36:56.474615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878ff00-3751-4549-b0eb-4f536103c4f1 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:36:56.474615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878ff00-3751-4549-b0eb-4f536103c4f1 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:36:56.475873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 753/1519] 10.222.0.47 () {54 vars in 961 bytes} [Thu Aug 4 04:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:36:56.480851 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092272f-09e8-4932-8d4a-7446725c3519 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:36:56.481275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092272f-09e8-4932-8d4a-7446725c3519 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:36:56.481408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092272f-09e8-4932-8d4a-7446725c3519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:36:56.502829 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-3092272f-09e8-4932-8d4a-7446725c3519 None None] MFA Rules not processed for user `350f72f740f749f68175c1c4e8ae9d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:36:56.503931 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3092272f-09e8-4932-8d4a-7446725c3519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:36:56.517096 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 767/1520] 10.222.0.47 () {58 vars in 1058 bytes} [Thu Aug 4 04:36:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:36:56.524822 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878ff00-3751-4549-b0eb-4f536103c4f1 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:36:56.539047 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878ff00-3751-4549-b0eb-4f536103c4f1 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:36:56.544895 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798b4d2-06dc-4d9c-be9a-d4d96ea9691e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350f72f740f749f68175c1c4e8ae9d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:36:56.545497 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798b4d2-06dc-4d9c-be9a-d4d96ea9691e service placement] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:36:56.546098 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798b4d2-06dc-4d9c-be9a-d4d96ea9691e service placement] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:36:56.546371 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798b4d2-06dc-4d9c-be9a-d4d96ea9691e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:36:56.554827 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798b4d2-06dc-4d9c-be9a-d4d96ea9691e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:36:56.556397 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798b4d2-06dc-4d9c-be9a-d4d96ea9691e service placement] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:36:56.566060 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 754/1521] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:36:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:37:03.576814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08743b2-7e08-4e13-b4c5-f04fcfb3ecba None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:37:03.577402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08743b2-7e08-4e13-b4c5-f04fcfb3ecba None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:37:03.577402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08743b2-7e08-4e13-b4c5-f04fcfb3ecba None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:37:03.578818 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 768/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 04:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:37:03.583967 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271df882-bcf8-406c-9a54-a703cb5ea727 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:37:03.584414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271df882-bcf8-406c-9a54-a703cb5ea727 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:37:03.584525 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271df882-bcf8-406c-9a54-a703cb5ea727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:37:03.604428 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-271df882-bcf8-406c-9a54-a703cb5ea727 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:37:03.605707 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-271df882-bcf8-406c-9a54-a703cb5ea727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:37:03.615006 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 755/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 04:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:37:03.773643 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08743b2-7e08-4e13-b4c5-f04fcfb3ecba None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:37:03.788795 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08743b2-7e08-4e13-b4c5-f04fcfb3ecba None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:37:03.794945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e9ef6-c598-44ae-b0d1-25a754a34b41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:37:03.795554 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e9ef6-c598-44ae-b0d1-25a754a34b41 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:37:03.796100 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e9ef6-c598-44ae-b0d1-25a754a34b41 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:37:03.796369 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e9ef6-c598-44ae-b0d1-25a754a34b41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:37:03.804848 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e9ef6-c598-44ae-b0d1-25a754a34b41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:37:03.806458 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e9ef6-c598-44ae-b0d1-25a754a34b41 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:37:03.817339 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 769/1524] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:37:04.650007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9f8b6-0ed7-4b53-aac6-a265970486c5 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:37:04.650635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9f8b6-0ed7-4b53-aac6-a265970486c5 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:37:04.650635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9f8b6-0ed7-4b53-aac6-a265970486c5 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:37:04.652751 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 756/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 04:37:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:37:04.658001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f6753-5b43-4647-a5d7-eaf9b4ca4efd None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:37:04.658522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f6753-5b43-4647-a5d7-eaf9b4ca4efd None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:37:04.658522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f6753-5b43-4647-a5d7-eaf9b4ca4efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:37:04.678806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7c2f6753-5b43-4647-a5d7-eaf9b4ca4efd None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:37:04.680051 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7c2f6753-5b43-4647-a5d7-eaf9b4ca4efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:37:04.688286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 770/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 04:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:37:04.841435 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9f8b6-0ed7-4b53-aac6-a265970486c5 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:37:04.856245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9f8b6-0ed7-4b53-aac6-a265970486c5 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:37:04.861849 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e11f3-b0df-49d5-b529-dbbe920bb73b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:37:04.862463 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e11f3-b0df-49d5-b529-dbbe920bb73b service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:37:04.862737 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e11f3-b0df-49d5-b529-dbbe920bb73b service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:37:04.863008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e11f3-b0df-49d5-b529-dbbe920bb73b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:37:04.871828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e11f3-b0df-49d5-b529-dbbe920bb73b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:37:04.873398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e11f3-b0df-49d5-b529-dbbe920bb73b service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:37:04.883206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 757/1527] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:37:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:37:08.838829 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba9dd0-b01e-4994-ad27-cd8f64445eb4 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:37:08.839491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba9dd0-b01e-4994-ad27-cd8f64445eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:37:08.839605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba9dd0-b01e-4994-ad27-cd8f64445eb4 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:37:08.840903 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 771/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 04:37:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:37:08.846319 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9070896-d158-4e38-9781-448035bb1ec7 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:37:08.846717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9070896-d158-4e38-9781-448035bb1ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:37:08.846872 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9070896-d158-4e38-9781-448035bb1ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:37:08.866464 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-c9070896-d158-4e38-9781-448035bb1ec7 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:37:08.867662 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c9070896-d158-4e38-9781-448035bb1ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:37:08.875649 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 758/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 04:37:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:37:08.949636 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba9dd0-b01e-4994-ad27-cd8f64445eb4 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:37:08.951351 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f9ba9dd0-b01e-4994-ad27-cd8f64445eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:37:09.029442 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba9dd0-b01e-4994-ad27-cd8f64445eb4 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:37:09.033097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c53b7-f206-4e9c-828f-6d41c3af8efb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:37:09.033738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c53b7-f206-4e9c-828f-6d41c3af8efb service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:37:09.034020 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c53b7-f206-4e9c-828f-6d41c3af8efb service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:37:09.034292 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c53b7-f206-4e9c-828f-6d41c3af8efb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:37:09.044034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3c53b7-f206-4e9c-828f-6d41c3af8efb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:37:09.045454 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3c53b7-f206-4e9c-828f-6d41c3af8efb service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:37:09.055671 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 772/1530] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:37:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:37:11.092641 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35151a2b-2111-4820-baab-8feec6f09b48 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:37:11.093222 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35151a2b-2111-4820-baab-8feec6f09b48 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:37:11.093330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35151a2b-2111-4820-baab-8feec6f09b48 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:37:11.094713 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 759/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 04:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:37:11.100149 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f9aac-14f2-4dd2-a1c6-d2f17fbb4b71 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:37:11.100569 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f9aac-14f2-4dd2-a1c6-d2f17fbb4b71 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:37:11.100684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f9aac-14f2-4dd2-a1c6-d2f17fbb4b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:37:11.121647 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-a31f9aac-14f2-4dd2-a1c6-d2f17fbb4b71 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:37:11.122784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a31f9aac-14f2-4dd2-a1c6-d2f17fbb4b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:37:11.130867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 773/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 04:37:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:37:11.168034 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35151a2b-2111-4820-baab-8feec6f09b48 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:37:11.179290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35151a2b-2111-4820-baab-8feec6f09b48 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:37:11.184724 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc26ca-bc4a-42e4-b68c-9beeacaf8dcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:37:11.185334 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc26ca-bc4a-42e4-b68c-9beeacaf8dcc service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:37:11.185643 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc26ca-bc4a-42e4-b68c-9beeacaf8dcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:37:11.185908 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc26ca-bc4a-42e4-b68c-9beeacaf8dcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:37:11.195327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc26ca-bc4a-42e4-b68c-9beeacaf8dcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:37:11.196736 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc26ca-bc4a-42e4-b68c-9beeacaf8dcc service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:37:11.207511 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 760/1533] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:37:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:37:14.726973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ac876-ba9d-41c3-83ea-d46e95033280 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:37:14.727515 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ac876-ba9d-41c3-83ea-d46e95033280 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:37:14.727515 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ac876-ba9d-41c3-83ea-d46e95033280 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:37:14.728877 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 774/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 04:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:37:14.734416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986c24b-cd55-4ffb-ba3a-d94125da1107 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:37:14.734883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986c24b-cd55-4ffb-ba3a-d94125da1107 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:37:14.734987 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986c24b-cd55-4ffb-ba3a-d94125da1107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:37:14.742628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=81 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:37:14.756156 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-f986c24b-cd55-4ffb-ba3a-d94125da1107 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:37:14.757295 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-f986c24b-cd55-4ffb-ba3a-d94125da1107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:37:14.766084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 761/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 04:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:37:15.075993 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ac876-ba9d-41c3-83ea-d46e95033280 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:37:15.078388 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e82ac876-ba9d-41c3-83ea-d46e95033280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:37:15.149199 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:37:15.164203 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ac876-ba9d-41c3-83ea-d46e95033280 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:37:15.168527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aca057-d5ea-4682-bf1d-66ccf8db12b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:37:15.169155 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aca057-d5ea-4682-bf1d-66ccf8db12b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:37:15.169427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aca057-d5ea-4682-bf1d-66ccf8db12b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:37:15.169738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aca057-d5ea-4682-bf1d-66ccf8db12b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:37:15.180264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aca057-d5ea-4682-bf1d-66ccf8db12b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:37:15.182341 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aca057-d5ea-4682-bf1d-66ccf8db12b1 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:37:15.192304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 775/1536] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:37:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:37:17.276874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503bf8c8-3ad5-4483-b5da-71d071eece5b None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:37:17.277469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503bf8c8-3ad5-4483-b5da-71d071eece5b None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:37:17.277469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503bf8c8-3ad5-4483-b5da-71d071eece5b None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:37:17.278837 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 762/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 04:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:37:17.284712 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fc9d3-2851-44eb-ad23-5209f11618a0 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:37:17.285230 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fc9d3-2851-44eb-ad23-5209f11618a0 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:37:17.285369 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fc9d3-2851-44eb-ad23-5209f11618a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:37:17.315909 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b87fc9d3-2851-44eb-ad23-5209f11618a0 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:37:17.317151 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b87fc9d3-2851-44eb-ad23-5209f11618a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:37:17.326171 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 776/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 04:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:37:17.637856 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503bf8c8-3ad5-4483-b5da-71d071eece5b None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:37:17.639543 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-503bf8c8-3ad5-4483-b5da-71d071eece5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:37:17.652860 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503bf8c8-3ad5-4483-b5da-71d071eece5b None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:37:17.656783 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d91b9-26f3-4f54-bfe1-f2851d201b0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:37:17.657382 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d91b9-26f3-4f54-bfe1-f2851d201b0d service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:37:17.657698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d91b9-26f3-4f54-bfe1-f2851d201b0d service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:37:17.657991 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d91b9-26f3-4f54-bfe1-f2851d201b0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:37:17.667945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d91b9-26f3-4f54-bfe1-f2851d201b0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:37:17.669352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d91b9-26f3-4f54-bfe1-f2851d201b0d service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:37:17.679413 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 763/1539] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:37:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:37:27.310620 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=11 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:37:27.675218 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=8 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:42.236099 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6dc87a-15b4-4834-8be9-351b37eb6c8c None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.236756 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6dc87a-15b4-4834-8be9-351b37eb6c8c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.236756 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6dc87a-15b4-4834-8be9-351b37eb6c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.264360 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-2e6dc87a-15b4-4834-8be9-351b37eb6c8c None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:42.267038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2e6dc87a-15b4-4834-8be9-351b37eb6c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:42.280631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 777/1540] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:42.291337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3e3e0-74d1-44e4-a61c-97153b3a489c None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.291925 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3e3e0-74d1-44e4-a61c-97153b3a489c None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.291925 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3e3e0-74d1-44e4-a61c-97153b3a489c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.313793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92784bc3-a81a-42f3-9f15-df6dc650b938 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.314088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92784bc3-a81a-42f3-9f15-df6dc650b938 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.314387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92784bc3-a81a-42f3-9f15-df6dc650b938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.330182 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-e9e3e3e0-74d1-44e4-a61c-97153b3a489c None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:42.331439 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-e9e3e3e0-74d1-44e4-a61c-97153b3a489c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:42.356534 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-92784bc3-a81a-42f3-9f15-df6dc650b938 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:42.358002 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-92784bc3-a81a-42f3-9f15-df6dc650b938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:42.361779 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 764/1541] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:42.367456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3462cf-6273-4dd7-97a9-8489599ca583 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.367778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3462cf-6273-4dd7-97a9-8489599ca583 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.368031 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3462cf-6273-4dd7-97a9-8489599ca583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.383153 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 778/1542] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:38:42.384028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92784bc3-a81a-42f3-9f15-df6dc650b938 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:42.399389 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-ff3462cf-6273-4dd7-97a9-8489599ca583 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:42.400642 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-ff3462cf-6273-4dd7-97a9-8489599ca583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:42.404703 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92784bc3-a81a-42f3-9f15-df6dc650b938 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:42.410045 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 765/1543] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:38:42.411491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5c50a-6f34-4e39-92b6-45ec00ef44b3 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.411880 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5c50a-6f34-4e39-92b6-45ec00ef44b3 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.412167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5c50a-6f34-4e39-92b6-45ec00ef44b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.422527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af368c4-98b2-4a95-8c2d-ab8648470679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:42.423216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af368c4-98b2-4a95-8c2d-ab8648470679 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.423495 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af368c4-98b2-4a95-8c2d-ab8648470679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.423764 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af368c4-98b2-4a95-8c2d-ab8648470679 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.424475 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af368c4-98b2-4a95-8c2d-ab8648470679 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:38:42.424774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af368c4-98b2-4a95-8c2d-ab8648470679 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:42.426717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af368c4-98b2-4a95-8c2d-ab8648470679 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:42.435238 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-3ff5c50a-6f34-4e39-92b6-45ec00ef44b3 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:42.436757 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-3ff5c50a-6f34-4e39-92b6-45ec00ef44b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:42.454516 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 779/1544] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:38:42.456602 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 766/1545] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:42.459695 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa194ef7-3d12-4e5c-b48b-66f3f0b4c68d None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.460569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa194ef7-3d12-4e5c-b48b-66f3f0b4c68d None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.460948 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a78dcf-8678-4182-9cc2-fb95f26933db None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.461274 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a78dcf-8678-4182-9cc2-fb95f26933db None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.461446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa194ef7-3d12-4e5c-b48b-66f3f0b4c68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.461597 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a78dcf-8678-4182-9cc2-fb95f26933db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.485606 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-fa194ef7-3d12-4e5c-b48b-66f3f0b4c68d None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:42.486870 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-fa194ef7-3d12-4e5c-b48b-66f3f0b4c68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:42.492445 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-57a78dcf-8678-4182-9cc2-fb95f26933db None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:42.493730 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-57a78dcf-8678-4182-9cc2-fb95f26933db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:42.495583 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 767/1546] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:42.496588 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4fe48-829a-45ba-850f-6c8a0368e7f8 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.496872 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4fe48-829a-45ba-850f-6c8a0368e7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.497141 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4fe48-829a-45ba-850f-6c8a0368e7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.505431 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 780/1547] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:42.506789 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e02b42-9359-4fb7-a928-2f00ab4a765d None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.507080 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e02b42-9359-4fb7-a928-2f00ab4a765d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.507353 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e02b42-9359-4fb7-a928-2f00ab4a765d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.529443 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-28a4fe48-829a-45ba-850f-6c8a0368e7f8 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:42.530705 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-28a4fe48-829a-45ba-850f-6c8a0368e7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:42.532906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-76e02b42-9359-4fb7-a928-2f00ab4a765d None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:42.536659 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-76e02b42-9359-4fb7-a928-2f00ab4a765d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:42.544596 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 768/1548] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:42.545709 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4fe48-829a-45ba-850f-6c8a0368e7f8 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:42.547328 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 781/1549] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:42.551378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b308918-a0ad-4793-91fd-c8ea5e9df1c9 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.551378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b308918-a0ad-4793-91fd-c8ea5e9df1c9 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.551378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b308918-a0ad-4793-91fd-c8ea5e9df1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.558445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4fe48-829a-45ba-850f-6c8a0368e7f8 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:42.563879 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ccb3d4-2cfe-43ed-a44f-bd22978d18d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:42.564675 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ccb3d4-2cfe-43ed-a44f-bd22978d18d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.564958 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ccb3d4-2cfe-43ed-a44f-bd22978d18d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.565227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ccb3d4-2cfe-43ed-a44f-bd22978d18d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.565930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ccb3d4-2cfe-43ed-a44f-bd22978d18d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:42.567961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ccb3d4-2cfe-43ed-a44f-bd22978d18d8 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:42.573977 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-6b308918-a0ad-4793-91fd-c8ea5e9df1c9 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:42.575510 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6b308918-a0ad-4793-91fd-c8ea5e9df1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:42.585922 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 782/1550] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:42.587711 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6801c5c-0e8f-49b7-82d0-eb6d3a83a3b7 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.588100 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6801c5c-0e8f-49b7-82d0-eb6d3a83a3b7 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.588474 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6801c5c-0e8f-49b7-82d0-eb6d3a83a3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.608600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 769/1551] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:38:42 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:42.608808 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-c6801c5c-0e8f-49b7-82d0-eb6d3a83a3b7 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:42.609957 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c6801c5c-0e8f-49b7-82d0-eb6d3a83a3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:42.610154 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ccb3d4-2cfe-43ed-a44f-bd22978d18d8 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:42.672011 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 783/1552] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:42.672921 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6801c5c-0e8f-49b7-82d0-eb6d3a83a3b7 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:42.681948 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ccb3d4-2cfe-43ed-a44f-bd22978d18d8 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:42.682232 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6801c5c-0e8f-49b7-82d0-eb6d3a83a3b7 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:42.686675 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f1b28-d66f-4b97-bad6-555e690a996b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:42.686837 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735b44c-a1cc-4ea6-8c8d-db56d6b466ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:42.687897 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f1b28-d66f-4b97-bad6-555e690a996b admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.688194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735b44c-a1cc-4ea6-8c8d-db56d6b466ff admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.688305 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f1b28-d66f-4b97-bad6-555e690a996b admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.688473 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735b44c-a1cc-4ea6-8c8d-db56d6b466ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.688597 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f1b28-d66f-4b97-bad6-555e690a996b admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.688737 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735b44c-a1cc-4ea6-8c8d-db56d6b466ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.689179 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f1b28-d66f-4b97-bad6-555e690a996b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:38:42.689419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735b44c-a1cc-4ea6-8c8d-db56d6b466ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:38:42.689516 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f1b28-d66f-4b97-bad6-555e690a996b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:42.689755 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735b44c-a1cc-4ea6-8c8d-db56d6b466ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:42.690498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: WARNING py.warnings [None req-943f1b28-d66f-4b97-bad6-555e690a996b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 04:38:42.690498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: warnings.warn(msg) Aug 04 04:38:42.690498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: Aug 04 04:38:42.691327 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735b44c-a1cc-4ea6-8c8d-db56d6b466ff admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:42.691436 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f1b28-d66f-4b97-bad6-555e690a996b admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:42.703928 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 770/1553] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:38:42.704914 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f1b28-d66f-4b97-bad6-555e690a996b admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:42.705521 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 784/1554] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:38:42.706868 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293e7a2-5c0b-452c-9c02-c657862b0753 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.707210 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293e7a2-5c0b-452c-9c02-c657862b0753 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.707481 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293e7a2-5c0b-452c-9c02-c657862b0753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.726706 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f1b28-d66f-4b97-bad6-555e690a996b admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:42.728229 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-c293e7a2-5c0b-452c-9c02-c657862b0753 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:42.729442 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c293e7a2-5c0b-452c-9c02-c657862b0753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:42.731457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318885a-04df-49b5-9813-e2311b16eade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:42.732068 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318885a-04df-49b5-9813-e2311b16eade admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.732342 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318885a-04df-49b5-9813-e2311b16eade admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.732611 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318885a-04df-49b5-9813-e2311b16eade admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.733300 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318885a-04df-49b5-9813-e2311b16eade admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:38:42.733631 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318885a-04df-49b5-9813-e2311b16eade admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:42.736004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318885a-04df-49b5-9813-e2311b16eade admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:42.738507 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 785/1555] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:42.739835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1b2a9-c778-462f-9ffb-a1cdb5a72764 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.740126 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1b2a9-c778-462f-9ffb-a1cdb5a72764 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.740400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1b2a9-c778-462f-9ffb-a1cdb5a72764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.750439 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 771/1556] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:38:42.751723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fde908-a889-476d-8ef1-061e7532a3ca None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.752011 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fde908-a889-476d-8ef1-061e7532a3ca None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.752419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fde908-a889-476d-8ef1-061e7532a3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.763396 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-73e1b2a9-c778-462f-9ffb-a1cdb5a72764 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:42.764599 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-73e1b2a9-c778-462f-9ffb-a1cdb5a72764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:42.771672 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-57fde908-a889-476d-8ef1-061e7532a3ca None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:42.772536 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 786/1557] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:42.772851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-57fde908-a889-476d-8ef1-061e7532a3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:42.773299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e1b2a9-c778-462f-9ffb-a1cdb5a72764 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:42.781256 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 772/1558] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:42.782868 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646bc29-6649-49c6-9c5b-2bad47643f42 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.783146 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646bc29-6649-49c6-9c5b-2bad47643f42 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.783415 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646bc29-6649-49c6-9c5b-2bad47643f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.787159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e1b2a9-c778-462f-9ffb-a1cdb5a72764 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:42.791771 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045bf698-84c1-459a-b4e3-0127a7587ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:42.792382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bf698-84c1-459a-b4e3-0127a7587ec2 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.792661 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bf698-84c1-459a-b4e3-0127a7587ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.792931 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bf698-84c1-459a-b4e3-0127a7587ec2 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.793618 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045bf698-84c1-459a-b4e3-0127a7587ec2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:42.795801 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045bf698-84c1-459a-b4e3-0127a7587ec2 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:42.803821 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-9646bc29-6649-49c6-9c5b-2bad47643f42 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:42.805016 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-9646bc29-6649-49c6-9c5b-2bad47643f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:42.813619 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 773/1559] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:42.814468 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646bc29-6649-49c6-9c5b-2bad47643f42 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:42.822527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 787/1560] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:38:42 2022] POST /identity/v3/users => generated 287 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:42.823611 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045bf698-84c1-459a-b4e3-0127a7587ec2 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:42.824102 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646bc29-6649-49c6-9c5b-2bad47643f42 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:42.829936 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b93254-44e0-499e-9a08-5fc05999541a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:42.830550 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b93254-44e0-499e-9a08-5fc05999541a admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.830835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b93254-44e0-499e-9a08-5fc05999541a admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.831109 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b93254-44e0-499e-9a08-5fc05999541a admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.831781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b93254-44e0-499e-9a08-5fc05999541a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:42.833733 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b93254-44e0-499e-9a08-5fc05999541a admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:42.834396 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045bf698-84c1-459a-b4e3-0127a7587ec2 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:42.839005 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d79f05-1d17-451e-86ca-aa3f59f7d0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:42.839607 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d79f05-1d17-451e-86ca-aa3f59f7d0f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.839882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d79f05-1d17-451e-86ca-aa3f59f7d0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.840149 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d79f05-1d17-451e-86ca-aa3f59f7d0f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.840803 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d79f05-1d17-451e-86ca-aa3f59f7d0f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:42.841848 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: WARNING py.warnings [None req-46d79f05-1d17-451e-86ca-aa3f59f7d0f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 04:38:42.841848 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: warnings.warn(msg) Aug 04 04:38:42.841848 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: Aug 04 04:38:42.843015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d79f05-1d17-451e-86ca-aa3f59f7d0f9 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:42.865570 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 774/1561] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:38:42 2022] POST /identity/v3/projects => generated 405 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:42.866651 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b93254-44e0-499e-9a08-5fc05999541a admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:42.875079 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 788/1562] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:38:42 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:42.878658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d79f05-1d17-451e-86ca-aa3f59f7d0f9 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:42.912496 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:42.928516 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:42.932703 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b93254-44e0-499e-9a08-5fc05999541a admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:42.943244 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d79f05-1d17-451e-86ca-aa3f59f7d0f9 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:42.947901 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d1f63-8237-4c43-bd4a-3b074336cb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:42.948520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d1f63-8237-4c43-bd4a-3b074336cb3c admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.948800 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d1f63-8237-4c43-bd4a-3b074336cb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.949075 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d1f63-8237-4c43-bd4a-3b074336cb3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.949778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d1f63-8237-4c43-bd4a-3b074336cb3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:42.951690 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d1f63-8237-4c43-bd4a-3b074336cb3c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:42.980473 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 789/1563] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:38:42 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:42.982440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9dd4e-f987-4f9d-a2ea-0da8da5ecfaf None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.982869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9dd4e-f987-4f9d-a2ea-0da8da5ecfaf None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.983249 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9dd4e-f987-4f9d-a2ea-0da8da5ecfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.993309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d1220-dc5c-470b-a9bc-4b7842101153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:42.994113 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d1220-dc5c-470b-a9bc-4b7842101153 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:42.994477 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d1220-dc5c-470b-a9bc-4b7842101153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:42.994923 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d1220-dc5c-470b-a9bc-4b7842101153 admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:42.995917 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d1220-dc5c-470b-a9bc-4b7842101153 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:38:42.996319 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d1220-dc5c-470b-a9bc-4b7842101153 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:42.998143 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d1220-dc5c-470b-a9bc-4b7842101153 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.004508 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b1d9dd4e-f987-4f9d-a2ea-0da8da5ecfaf None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:43.005737 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b1d9dd4e-f987-4f9d-a2ea-0da8da5ecfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:43.011183 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 775/1564] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:38:43.012036 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d1220-dc5c-470b-a9bc-4b7842101153 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.025028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d1220-dc5c-470b-a9bc-4b7842101153 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.066770 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 790/1565] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:43.067561 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9dd4e-f987-4f9d-a2ea-0da8da5ecfaf None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.077011 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9dd4e-f987-4f9d-a2ea-0da8da5ecfaf None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.082320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f11c85-68e0-45ef-8009-4698dd2cbdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.082951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f11c85-68e0-45ef-8009-4698dd2cbdfe admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.083306 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f11c85-68e0-45ef-8009-4698dd2cbdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.083497 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f11c85-68e0-45ef-8009-4698dd2cbdfe admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.083937 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f503ed-1078-4be4-9e5d-d7a6323b58d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.084185 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f11c85-68e0-45ef-8009-4698dd2cbdfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:38:43.084484 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f11c85-68e0-45ef-8009-4698dd2cbdfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.084584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f503ed-1078-4be4-9e5d-d7a6323b58d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.084824 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f503ed-1078-4be4-9e5d-d7a6323b58d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.085093 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f503ed-1078-4be4-9e5d-d7a6323b58d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.085813 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f503ed-1078-4be4-9e5d-d7a6323b58d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:38:43.086108 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f11c85-68e0-45ef-8009-4698dd2cbdfe admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.086218 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f503ed-1078-4be4-9e5d-d7a6323b58d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.087728 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f503ed-1078-4be4-9e5d-d7a6323b58d0 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.098583 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 791/1566] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:38:43.099419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f11c85-68e0-45ef-8009-4698dd2cbdfe admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.109155 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 776/1567] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:38:43.109155 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f503ed-1078-4be4-9e5d-d7a6323b58d0 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.111492 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f11c85-68e0-45ef-8009-4698dd2cbdfe admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.116195 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf8d71-747c-41da-8250-992716d7a5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.116738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf8d71-747c-41da-8250-992716d7a5b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.117017 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf8d71-747c-41da-8250-992716d7a5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.117288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf8d71-747c-41da-8250-992716d7a5b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.118023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf8d71-747c-41da-8250-992716d7a5b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.119036 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f503ed-1078-4be4-9e5d-d7a6323b58d0 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.119612 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf8d71-747c-41da-8250-992716d7a5b1 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.125040 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9e3d9-06ff-4b21-b13a-919f9eb0b37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.125727 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9e3d9-06ff-4b21-b13a-919f9eb0b37e admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.125997 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9e3d9-06ff-4b21-b13a-919f9eb0b37e admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.126265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9e3d9-06ff-4b21-b13a-919f9eb0b37e admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.126938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9e3d9-06ff-4b21-b13a-919f9eb0b37e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.127970 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: WARNING py.warnings [None req-00d9e3d9-06ff-4b21-b13a-919f9eb0b37e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 04:38:43.127970 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: warnings.warn(msg) Aug 04 04:38:43.127970 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: Aug 04 04:38:43.129157 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9e3d9-06ff-4b21-b13a-919f9eb0b37e admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.132603 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 792/1568] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:43.133273 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf8d71-747c-41da-8250-992716d7a5b1 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.144949 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf8d71-747c-41da-8250-992716d7a5b1 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.152510 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ee1d2-2d72-4ef2-ac6d-4ef92e82db84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.152510 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ee1d2-2d72-4ef2-ac6d-4ef92e82db84 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.152510 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ee1d2-2d72-4ef2-ac6d-4ef92e82db84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.152510 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ee1d2-2d72-4ef2-ac6d-4ef92e82db84 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.153029 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ee1d2-2d72-4ef2-ac6d-4ef92e82db84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.155173 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ee1d2-2d72-4ef2-ac6d-4ef92e82db84 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.160066 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 777/1569] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:38:43 2022] POST /identity/v3/users => generated 303 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:43.160066 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9e3d9-06ff-4b21-b13a-919f9eb0b37e admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.167952 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9e3d9-06ff-4b21-b13a-919f9eb0b37e admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.174343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f7d69-6a92-4fdf-92de-3c3347cc3fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.174969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f7d69-6a92-4fdf-92de-3c3347cc3fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.175245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f7d69-6a92-4fdf-92de-3c3347cc3fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.175516 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f7d69-6a92-4fdf-92de-3c3347cc3fc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.176182 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f7d69-6a92-4fdf-92de-3c3347cc3fc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.178197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f7d69-6a92-4fdf-92de-3c3347cc3fc7 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.178338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 793/1570] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:38:43 2022] POST /identity/v3/users => generated 293 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:43.181532 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ee1d2-2d72-4ef2-ac6d-4ef92e82db84 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.190383 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ee1d2-2d72-4ef2-ac6d-4ef92e82db84 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.197726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974f256-0762-40f4-88ec-6be5c3b957fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.198482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974f256-0762-40f4-88ec-6be5c3b957fa admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.198761 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974f256-0762-40f4-88ec-6be5c3b957fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.199030 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974f256-0762-40f4-88ec-6be5c3b957fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.199709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974f256-0762-40f4-88ec-6be5c3b957fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.201705 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974f256-0762-40f4-88ec-6be5c3b957fa admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.203776 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 778/1571] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:38:43 2022] POST /identity/v3/users => generated 290 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:43.204407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f7d69-6a92-4fdf-92de-3c3347cc3fc7 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.214735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f7d69-6a92-4fdf-92de-3c3347cc3fc7 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.220412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2fee0-8879-4296-a7fd-be4cb9cbf471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.221317 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2fee0-8879-4296-a7fd-be4cb9cbf471 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.221753 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2fee0-8879-4296-a7fd-be4cb9cbf471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.222132 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2fee0-8879-4296-a7fd-be4cb9cbf471 admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.223159 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d2fee0-8879-4296-a7fd-be4cb9cbf471 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:38:43.223621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d2fee0-8879-4296-a7fd-be4cb9cbf471 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.226282 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d2fee0-8879-4296-a7fd-be4cb9cbf471 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.238978 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 794/1572] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:38:43 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:43.247122 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974f256-0762-40f4-88ec-6be5c3b957fa admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.247302 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 779/1573] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:38:43.247302 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2fee0-8879-4296-a7fd-be4cb9cbf471 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.254393 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2fee0-8879-4296-a7fd-be4cb9cbf471 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.284945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:43.316565 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974f256-0762-40f4-88ec-6be5c3b957fa admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.318625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8da3a4-96dd-456b-a357-771f4307e609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.319475 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8da3a4-96dd-456b-a357-771f4307e609 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.319868 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8da3a4-96dd-456b-a357-771f4307e609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.320238 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8da3a4-96dd-456b-a357-771f4307e609 admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.320998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8da3a4-96dd-456b-a357-771f4307e609 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.321261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf5fcc-8e41-4774-846d-7d57e50db234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.321913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf5fcc-8e41-4774-846d-7d57e50db234 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.322201 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf5fcc-8e41-4774-846d-7d57e50db234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.322479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf5fcc-8e41-4774-846d-7d57e50db234 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.323150 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf5fcc-8e41-4774-846d-7d57e50db234 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.323474 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8da3a4-96dd-456b-a357-771f4307e609 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.325060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf5fcc-8e41-4774-846d-7d57e50db234 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.341185 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:43.358501 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 780/1574] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:38:43 2022] POST /identity/v3/projects => generated 357 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:43.358707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 795/1575] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:38:43 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:43.358871 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8da3a4-96dd-456b-a357-771f4307e609 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.360044 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf5fcc-8e41-4774-846d-7d57e50db234 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.422477 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8da3a4-96dd-456b-a357-771f4307e609 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.427506 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39b858-eb4e-4c92-9f61-4b0f0a95e945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.427674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf5fcc-8e41-4774-846d-7d57e50db234 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.428134 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39b858-eb4e-4c92-9f61-4b0f0a95e945 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.428414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39b858-eb4e-4c92-9f61-4b0f0a95e945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.428690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39b858-eb4e-4c92-9f61-4b0f0a95e945 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.429396 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39b858-eb4e-4c92-9f61-4b0f0a95e945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.431019 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39b858-eb4e-4c92-9f61-4b0f0a95e945 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.432386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec2e19-72eb-4339-927d-2ca4212b6628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.433077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec2e19-72eb-4339-927d-2ca4212b6628 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.433371 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec2e19-72eb-4339-927d-2ca4212b6628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.433693 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec2e19-72eb-4339-927d-2ca4212b6628 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.434430 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec2e19-72eb-4339-927d-2ca4212b6628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.436030 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec2e19-72eb-4339-927d-2ca4212b6628 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.442194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 781/1576] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:43.443039 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39b858-eb4e-4c92-9f61-4b0f0a95e945 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.453403 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 796/1577] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:43.453403 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec2e19-72eb-4339-927d-2ca4212b6628 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.454357 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39b858-eb4e-4c92-9f61-4b0f0a95e945 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.461601 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559a979-9e0a-4b74-9656-b3e74d67ed32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.462223 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559a979-9e0a-4b74-9656-b3e74d67ed32 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.462502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559a979-9e0a-4b74-9656-b3e74d67ed32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.462770 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559a979-9e0a-4b74-9656-b3e74d67ed32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.464752 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559a979-9e0a-4b74-9656-b3e74d67ed32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.464890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec2e19-72eb-4339-927d-2ca4212b6628 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.465017 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559a979-9e0a-4b74-9656-b3e74d67ed32 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.469363 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4da331-ec96-4a11-91bf-62f0d35aa3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.470031 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4da331-ec96-4a11-91bf-62f0d35aa3de admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.471807 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4da331-ec96-4a11-91bf-62f0d35aa3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.471807 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4da331-ec96-4a11-91bf-62f0d35aa3de admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.471807 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4da331-ec96-4a11-91bf-62f0d35aa3de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.473637 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4da331-ec96-4a11-91bf-62f0d35aa3de admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.478645 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 782/1578] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:43.479541 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559a979-9e0a-4b74-9656-b3e74d67ed32 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.487414 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 797/1579] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:43.488252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4da331-ec96-4a11-91bf-62f0d35aa3de admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.489799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559a979-9e0a-4b74-9656-b3e74d67ed32 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.494586 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f6cc7-c861-4515-8704-7f12759a6675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.495194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f6cc7-c861-4515-8704-7f12759a6675 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.495467 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f6cc7-c861-4515-8704-7f12759a6675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.495731 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f6cc7-c861-4515-8704-7f12759a6675 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.496386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06f6cc7-c861-4515-8704-7f12759a6675 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.499390 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06f6cc7-c861-4515-8704-7f12759a6675 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.499592 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4da331-ec96-4a11-91bf-62f0d35aa3de admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.508411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db91d59-f228-4a2c-a892-053bdc564b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.509041 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db91d59-f228-4a2c-a892-053bdc564b33 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.509528 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db91d59-f228-4a2c-a892-053bdc564b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.509701 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db91d59-f228-4a2c-a892-053bdc564b33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.510309 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db91d59-f228-4a2c-a892-053bdc564b33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.512281 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db91d59-f228-4a2c-a892-053bdc564b33 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.525957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 783/1580] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:38:43 2022] POST /identity/v3/users => generated 289 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:43.526770 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f6cc7-c861-4515-8704-7f12759a6675 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.539115 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f6cc7-c861-4515-8704-7f12759a6675 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.546063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c1f40-3120-4257-bd04-d004b64ef58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.546063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c1f40-3120-4257-bd04-d004b64ef58e admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.546063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c1f40-3120-4257-bd04-d004b64ef58e admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.546063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c1f40-3120-4257-bd04-d004b64ef58e admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.547180 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c1f40-3120-4257-bd04-d004b64ef58e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.549130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c1f40-3120-4257-bd04-d004b64ef58e admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.550270 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 798/1581] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:38:43 2022] POST /identity/v3/projects => generated 367 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:43.551184 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db91d59-f228-4a2c-a892-053bdc564b33 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.575358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 784/1582] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:38:43 2022] POST /identity/v3/users => generated 292 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:43.575358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c1f40-3120-4257-bd04-d004b64ef58e admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.586234 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c1f40-3120-4257-bd04-d004b64ef58e admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.620639 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db91d59-f228-4a2c-a892-053bdc564b33 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.626323 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863641d-023b-4245-92df-0e84037236b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.626959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863641d-023b-4245-92df-0e84037236b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.627299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863641d-023b-4245-92df-0e84037236b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.627601 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863641d-023b-4245-92df-0e84037236b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.628265 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c863641d-023b-4245-92df-0e84037236b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.630246 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c863641d-023b-4245-92df-0e84037236b7 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.648130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d99b5d9-98c9-4be5-9205-9d248f5fc81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.649041 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99b5d9-98c9-4be5-9205-9d248f5fc81b admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.649421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99b5d9-98c9-4be5-9205-9d248f5fc81b admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.649734 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99b5d9-98c9-4be5-9205-9d248f5fc81b admin admin] PATH_INFO: `/v3/projects/26ffb724e028422eb6f69be51973fe9b/users/85b8ce0f906e4f1796ca66fb4b98fc2b/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.652606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 799/1583] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:38:43 2022] POST /identity/v3/users => generated 279 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:43.653296 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863641d-023b-4245-92df-0e84037236b7 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.661860 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:43.663308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863641d-023b-4245-92df-0e84037236b7 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.668379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd09fa-6958-49cf-9b9e-5819792aa5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.669165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd09fa-6958-49cf-9b9e-5819792aa5c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.669456 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd09fa-6958-49cf-9b9e-5819792aa5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.670030 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd09fa-6958-49cf-9b9e-5819792aa5c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.670362 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d99b5d9-98c9-4be5-9205-9d248f5fc81b admin admin] RBAC: Authorizing `identity:create_grant(project_id=26ffb724e028422eb6f69be51973fe9b, user_id=85b8ce0f906e4f1796ca66fb4b98fc2b, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.670742 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd09fa-6958-49cf-9b9e-5819792aa5c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.672956 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd09fa-6958-49cf-9b9e-5819792aa5c6 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.674758 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d99b5d9-98c9-4be5-9205-9d248f5fc81b admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.684797 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 800/1584] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:43.685832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd09fa-6958-49cf-9b9e-5819792aa5c6 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.693876 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 785/1585] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:38:43 2022] PUT /identity/v3/projects/26ffb724e028422eb6f69be51973fe9b/users/85b8ce0f906e4f1796ca66fb4b98fc2b/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:38:43.694717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d99b5d9-98c9-4be5-9205-9d248f5fc81b admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.697054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd09fa-6958-49cf-9b9e-5819792aa5c6 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.738768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:43.764153 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21ae07-e6d1-41b9-a04d-aab9528be396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.764902 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21ae07-e6d1-41b9-a04d-aab9528be396 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.766462 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21ae07-e6d1-41b9-a04d-aab9528be396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.766803 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21ae07-e6d1-41b9-a04d-aab9528be396 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.767510 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21ae07-e6d1-41b9-a04d-aab9528be396 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.769144 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21ae07-e6d1-41b9-a04d-aab9528be396 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.770229 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d99b5d9-98c9-4be5-9205-9d248f5fc81b admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.775310 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a62ab6-ab8b-4570-949c-f54bf985bbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.775932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a62ab6-ab8b-4570-949c-f54bf985bbe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.776213 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a62ab6-ab8b-4570-949c-f54bf985bbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.776487 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a62ab6-ab8b-4570-949c-f54bf985bbe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.777181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a62ab6-ab8b-4570-949c-f54bf985bbe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.778801 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a62ab6-ab8b-4570-949c-f54bf985bbe4 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.779751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 801/1586] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:43.784628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21ae07-e6d1-41b9-a04d-aab9528be396 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.790912 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 786/1587] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:43.791594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a62ab6-ab8b-4570-949c-f54bf985bbe4 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.795818 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21ae07-e6d1-41b9-a04d-aab9528be396 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.800837 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31d164-9aea-458f-9375-cca4e8679e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.801458 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31d164-9aea-458f-9375-cca4e8679e85 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.801772 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31d164-9aea-458f-9375-cca4e8679e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.802041 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31d164-9aea-458f-9375-cca4e8679e85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.802141 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a62ab6-ab8b-4570-949c-f54bf985bbe4 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.802770 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31d164-9aea-458f-9375-cca4e8679e85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.804512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31d164-9aea-458f-9375-cca4e8679e85 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.806698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0f8e3-2682-46cb-a9cc-8b0126118aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.807307 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0f8e3-2682-46cb-a9cc-8b0126118aa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.807580 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0f8e3-2682-46cb-a9cc-8b0126118aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.807845 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0f8e3-2682-46cb-a9cc-8b0126118aa2 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.808503 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f0f8e3-2682-46cb-a9cc-8b0126118aa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.810415 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f0f8e3-2682-46cb-a9cc-8b0126118aa2 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.815379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 802/1588] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:43.816288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31d164-9aea-458f-9375-cca4e8679e85 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.827361 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31d164-9aea-458f-9375-cca4e8679e85 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.832610 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63845c45-de4c-4096-98bd-6f339382bca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.833263 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63845c45-de4c-4096-98bd-6f339382bca3 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.833500 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63845c45-de4c-4096-98bd-6f339382bca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.833807 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63845c45-de4c-4096-98bd-6f339382bca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.834505 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63845c45-de4c-4096-98bd-6f339382bca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.835072 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 787/1589] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:38:43 2022] POST /identity/v3/users => generated 284 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:43.835830 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0f8e3-2682-46cb-a9cc-8b0126118aa2 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.836250 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63845c45-de4c-4096-98bd-6f339382bca3 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.846903 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0f8e3-2682-46cb-a9cc-8b0126118aa2 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.847816 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 803/1590] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:43.848738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63845c45-de4c-4096-98bd-6f339382bca3 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.855939 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500ec94-19ec-474b-aaca-a17130a1fc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.856569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500ec94-19ec-474b-aaca-a17130a1fc02 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.856850 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500ec94-19ec-474b-aaca-a17130a1fc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.857121 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500ec94-19ec-474b-aaca-a17130a1fc02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.857867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d500ec94-19ec-474b-aaca-a17130a1fc02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.859294 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63845c45-de4c-4096-98bd-6f339382bca3 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:43.859512 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d500ec94-19ec-474b-aaca-a17130a1fc02 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.864639 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce155d12-5eb7-4977-820d-89ecea456931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:43.865320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce155d12-5eb7-4977-820d-89ecea456931 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.865641 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce155d12-5eb7-4977-820d-89ecea456931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.865921 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce155d12-5eb7-4977-820d-89ecea456931 admin admin] PATH_INFO: `/v3/projects/f3b4625caa104a7ebb014b7fa76494a3/users/714e4bd548f144959cdd903d226d7c3b/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.871105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 788/1591] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:43.871105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6075d-477a-41e8-86f9-5914e1e6f8ae None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:43.871105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6075d-477a-41e8-86f9-5914e1e6f8ae None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:43.871105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6075d-477a-41e8-86f9-5914e1e6f8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:43.884393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce155d12-5eb7-4977-820d-89ecea456931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3b4625caa104a7ebb014b7fa76494a3, user_id=714e4bd548f144959cdd903d226d7c3b, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:43.885463 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: WARNING py.warnings [None req-ce155d12-5eb7-4977-820d-89ecea456931 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 04:38:43.885463 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: warnings.warn(msg) Aug 04 04:38:43.885463 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: Aug 04 04:38:43.887970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce155d12-5eb7-4977-820d-89ecea456931 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:43.903000 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-7ab6075d-477a-41e8-86f9-5914e1e6f8ae None None] MFA Rules not processed for user `85b8ce0f906e4f1796ca66fb4b98fc2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:43.904220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-7ab6075d-477a-41e8-86f9-5914e1e6f8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:43.904840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 804/1592] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:38:43 2022] PUT /identity/v3/projects/f3b4625caa104a7ebb014b7fa76494a3/users/714e4bd548f144959cdd903d226d7c3b/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:38:43.910670 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce155d12-5eb7-4977-820d-89ecea456931 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.978398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 789/1593] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:43.979263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6075d-477a-41e8-86f9-5914e1e6f8ae None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:43.994908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce155d12-5eb7-4977-820d-89ecea456931 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:44.002789 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a4bf5-6d4a-47a7-ab80-f54643726f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:44.005388 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a4bf5-6d4a-47a7-ab80-f54643726f23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:44.005388 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a4bf5-6d4a-47a7-ab80-f54643726f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:44.005388 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a4bf5-6d4a-47a7-ab80-f54643726f23 admin admin] PATH_INFO: `/v3/projects/9124d19d48f44fe2a904a715e5db1223/users/49a7ce9a27734dc0a187d1b217206785/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:44.028399 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a4bf5-6d4a-47a7-ab80-f54643726f23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9124d19d48f44fe2a904a715e5db1223, user_id=49a7ce9a27734dc0a187d1b217206785, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:44.031892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a4bf5-6d4a-47a7-ab80-f54643726f23 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:44.038214 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:44.050240 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 805/1594] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:38:43 2022] PUT /identity/v3/projects/9124d19d48f44fe2a904a715e5db1223/users/49a7ce9a27734dc0a187d1b217206785/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:38:44.052287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a4bf5-6d4a-47a7-ab80-f54643726f23 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:44.058872 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6075d-477a-41e8-86f9-5914e1e6f8ae None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:44.124112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa62be3-a302-494a-9397-b34741edaae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:44.125588 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa62be3-a302-494a-9397-b34741edaae1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:44.125848 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a4bf5-6d4a-47a7-ab80-f54643726f23 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:44.126040 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa62be3-a302-494a-9397-b34741edaae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:44.126331 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa62be3-a302-494a-9397-b34741edaae1 admin admin] PATH_INFO: `/v3/projects/a89819385e8b42bb89e40449dd21ca86/users/99d00c09d8374f59ac1260ade95be9ec/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:44.130832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0218bdd6-f82d-4087-b4e2-5b64e4cf9cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:44.131465 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218bdd6-f82d-4087-b4e2-5b64e4cf9cbd admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:44.131751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218bdd6-f82d-4087-b4e2-5b64e4cf9cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:44.132025 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218bdd6-f82d-4087-b4e2-5b64e4cf9cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:44.132858 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0218bdd6-f82d-4087-b4e2-5b64e4cf9cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:44.134507 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0218bdd6-f82d-4087-b4e2-5b64e4cf9cbd admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:44.142886 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa62be3-a302-494a-9397-b34741edaae1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a89819385e8b42bb89e40449dd21ca86, user_id=99d00c09d8374f59ac1260ade95be9ec, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:44.145676 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa62be3-a302-494a-9397-b34741edaae1 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:44.146177 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 806/1595] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:44 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:44.147023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0218bdd6-f82d-4087-b4e2-5b64e4cf9cbd admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:44.158968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0218bdd6-f82d-4087-b4e2-5b64e4cf9cbd admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:44.162935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:44.164857 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 790/1596] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:38:43 2022] PUT /identity/v3/projects/a89819385e8b42bb89e40449dd21ca86/users/99d00c09d8374f59ac1260ade95be9ec/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:38:44.165644 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa62be3-a302-494a-9397-b34741edaae1 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:44.221794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7730930-399d-4cb7-acd7-f66bd6d3c3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:44.222429 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7730930-399d-4cb7-acd7-f66bd6d3c3c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:44.222710 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7730930-399d-4cb7-acd7-f66bd6d3c3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:44.222994 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7730930-399d-4cb7-acd7-f66bd6d3c3c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:44.223704 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7730930-399d-4cb7-acd7-f66bd6d3c3c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:44.225319 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7730930-399d-4cb7-acd7-f66bd6d3c3c5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:44.236684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa62be3-a302-494a-9397-b34741edaae1 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:44.237186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 807/1597] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:44 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:44.238145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7730930-399d-4cb7-acd7-f66bd6d3c3c5 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:44.241926 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55b9a0-844a-4f61-9529-9164d3ad67de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:44.242558 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55b9a0-844a-4f61-9529-9164d3ad67de admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:44.242840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55b9a0-844a-4f61-9529-9164d3ad67de admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:44.243162 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55b9a0-844a-4f61-9529-9164d3ad67de admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:44.243872 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e55b9a0-844a-4f61-9529-9164d3ad67de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:44.245490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e55b9a0-844a-4f61-9529-9164d3ad67de admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:44.248138 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7730930-399d-4cb7-acd7-f66bd6d3c3c5 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:44.254058 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96663951-9d7b-45ea-a4e4-149a8c905a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:44.254685 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96663951-9d7b-45ea-a4e4-149a8c905a23 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:44.254970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96663951-9d7b-45ea-a4e4-149a8c905a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:44.255245 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96663951-9d7b-45ea-a4e4-149a8c905a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:44.255987 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96663951-9d7b-45ea-a4e4-149a8c905a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:44.258135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 791/1598] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:44 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:44.258272 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96663951-9d7b-45ea-a4e4-149a8c905a23 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:44.258701 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042a4ad-da33-4222-8427-ed160e9767b6 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:44.258997 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042a4ad-da33-4222-8427-ed160e9767b6 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:44.259276 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042a4ad-da33-4222-8427-ed160e9767b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:44.274057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 808/1599] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:44.277760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ad974-bb57-47de-800b-0ab425b70fe2 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:44.277760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ad974-bb57-47de-800b-0ab425b70fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:44.277760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ad974-bb57-47de-800b-0ab425b70fe2 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:44.277760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 809/1600] 10.222.0.47 () {54 vars in 962 bytes} [Thu Aug 4 04:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:38:44.278536 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2ff30-ab77-448d-a350-51677c7d05ea None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:44.278695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2ff30-ab77-448d-a350-51677c7d05ea None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:44.278973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2ff30-ab77-448d-a350-51677c7d05ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:44.294436 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-d042a4ad-da33-4222-8427-ed160e9767b6 None None] MFA Rules not processed for user `714e4bd548f144959cdd903d226d7c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:44.296040 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d042a4ad-da33-4222-8427-ed160e9767b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:44.299458 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-bbf2ff30-ab77-448d-a350-51677c7d05ea None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:44.300655 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-bbf2ff30-ab77-448d-a350-51677c7d05ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:44.308423 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 810/1601] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:44.309308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf2ff30-ab77-448d-a350-51677c7d05ea None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:44.320677 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf2ff30-ab77-448d-a350-51677c7d05ea None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:44.325383 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31bd47c-142e-4263-94e7-d017a9722197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:44.326094 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31bd47c-142e-4263-94e7-d017a9722197 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:44.326377 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31bd47c-142e-4263-94e7-d017a9722197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:44.326649 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31bd47c-142e-4263-94e7-d017a9722197 admin admin] PATH_INFO: `/v3/projects/157a41f0f76c41cb8dfff71a14138534/users/12f7b1c19d824ac7b8207bf001b79057/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:44.342736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31bd47c-142e-4263-94e7-d017a9722197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=157a41f0f76c41cb8dfff71a14138534, user_id=12f7b1c19d824ac7b8207bf001b79057, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:44.345807 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31bd47c-142e-4263-94e7-d017a9722197 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:44.360546 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 811/1602] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:38:44 2022] PUT /identity/v3/projects/157a41f0f76c41cb8dfff71a14138534/users/12f7b1c19d824ac7b8207bf001b79057/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:38:44.361858 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf018e41-5534-4e34-b94a-cff17e69636a None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:44.362138 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf018e41-5534-4e34-b94a-cff17e69636a None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:44.362415 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf018e41-5534-4e34-b94a-cff17e69636a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:44.369706 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:44.375937 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 792/1603] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:44 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:44.377254 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042a4ad-da33-4222-8427-ed160e9767b6 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:44.383891 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-bf018e41-5534-4e34-b94a-cff17e69636a None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:44.385091 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-bf018e41-5534-4e34-b94a-cff17e69636a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:44.390815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042a4ad-da33-4222-8427-ed160e9767b6 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:44.449437 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 812/1604] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:44.450329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf018e41-5534-4e34-b94a-cff17e69636a None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:44.453790 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a3c9f-764a-40fd-ae5b-984521358019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:44.454438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a3c9f-764a-40fd-ae5b-984521358019 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:44.454716 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a3c9f-764a-40fd-ae5b-984521358019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:44.454988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a3c9f-764a-40fd-ae5b-984521358019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:44.455704 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a3c9f-764a-40fd-ae5b-984521358019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:44.457321 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a3c9f-764a-40fd-ae5b-984521358019 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:44.461417 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf018e41-5534-4e34-b94a-cff17e69636a None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:44.468222 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 793/1605] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:44 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:44.468979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36d7ee-3073-479e-8bb2-940622ed8cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:44.470269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a3c9f-764a-40fd-ae5b-984521358019 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:44.470444 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36d7ee-3073-479e-8bb2-940622ed8cc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:44.470444 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36d7ee-3073-479e-8bb2-940622ed8cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:44.470444 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36d7ee-3073-479e-8bb2-940622ed8cc6 admin admin] PATH_INFO: `/v3/projects/3c72f723819849228c8763c36ded1d11/users/fa8866699789473ca655509b6c3e6fe6/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:44.482161 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a3c9f-764a-40fd-ae5b-984521358019 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:44.486784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36d7ee-3073-479e-8bb2-940622ed8cc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c72f723819849228c8763c36ded1d11, user_id=fa8866699789473ca655509b6c3e6fe6, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:44.487599 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2dcfa6-5df4-41b5-9fcc-2329a0373645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:44.488277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2dcfa6-5df4-41b5-9fcc-2329a0373645 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:44.488540 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2dcfa6-5df4-41b5-9fcc-2329a0373645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:44.488813 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2dcfa6-5df4-41b5-9fcc-2329a0373645 admin admin] PATH_INFO: `/v3/projects/9fcc6257d02c4927b30006a3a6c980ae/users/dfe222e6e5f84b70845df382889a14f5/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:44.489416 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36d7ee-3073-479e-8bb2-940622ed8cc6 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:44.507675 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2dcfa6-5df4-41b5-9fcc-2329a0373645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fcc6257d02c4927b30006a3a6c980ae, user_id=dfe222e6e5f84b70845df382889a14f5, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:44.509012 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 813/1606] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:38:44 2022] PUT /identity/v3/projects/3c72f723819849228c8763c36ded1d11/users/fa8866699789473ca655509b6c3e6fe6/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:38:44.513727 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2af66-fa9f-4b88-8219-1b674e252a57 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:44.513727 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2af66-fa9f-4b88-8219-1b674e252a57 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:44.514027 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2dcfa6-5df4-41b5-9fcc-2329a0373645 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:44.514132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2af66-fa9f-4b88-8219-1b674e252a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:44.533889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 794/1607] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:38:44 2022] PUT /identity/v3/projects/9fcc6257d02c4927b30006a3a6c980ae/users/dfe222e6e5f84b70845df382889a14f5/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:38:44.534886 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2dcfa6-5df4-41b5-9fcc-2329a0373645 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:44.557215 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-26c2af66-fa9f-4b88-8219-1b674e252a57 None None] MFA Rules not processed for user `caa2901451aa4554bdfcac1bddbc620a`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:44.559509 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-26c2af66-fa9f-4b88-8219-1b674e252a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:44.579509 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:44.596223 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2dcfa6-5df4-41b5-9fcc-2329a0373645 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:44.600819 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4361f-9363-4a21-a1f8-9da632b145b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:44.601446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4361f-9363-4a21-a1f8-9da632b145b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:44.601769 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4361f-9363-4a21-a1f8-9da632b145b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:44.602040 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4361f-9363-4a21-a1f8-9da632b145b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:44.632976 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 814/1608] 10.222.0.47 () {58 vars in 1059 bytes} [Thu Aug 4 04:38:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:44.634500 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926b65a-24b4-4ea7-807d-e5dfbb0f0d23 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:44.634795 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926b65a-24b4-4ea7-807d-e5dfbb0f0d23 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:44.635075 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926b65a-24b4-4ea7-807d-e5dfbb0f0d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:44.654284 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7926b65a-24b4-4ea7-807d-e5dfbb0f0d23 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:44.655502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7926b65a-24b4-4ea7-807d-e5dfbb0f0d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:44.676440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca4361f-9363-4a21-a1f8-9da632b145b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:44.678366 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca4361f-9363-4a21-a1f8-9da632b145b9 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:44.690666 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 795/1609] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:38:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:38:44.691735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4361f-9363-4a21-a1f8-9da632b145b9 service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:44.701572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4361f-9363-4a21-a1f8-9da632b145b9 service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:44.710577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308f333-6059-406c-a840-c0148ede939f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:44.711218 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308f333-6059-406c-a840-c0148ede939f admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:44.711500 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308f333-6059-406c-a840-c0148ede939f admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:44.711788 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308f333-6059-406c-a840-c0148ede939f admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:44.712620 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308f333-6059-406c-a840-c0148ede939f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:44.714484 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308f333-6059-406c-a840-c0148ede939f admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:44.729044 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 815/1610] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:44.729044 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926b65a-24b4-4ea7-807d-e5dfbb0f0d23 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:44.741287 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:44.744729 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926b65a-24b4-4ea7-807d-e5dfbb0f0d23 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:44.754138 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b33472-d545-42a1-91fb-67748e4f17ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:44.755054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b33472-d545-42a1-91fb-67748e4f17ba service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:44.755397 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b33472-d545-42a1-91fb-67748e4f17ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:44.755716 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b33472-d545-42a1-91fb-67748e4f17ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:44.763741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 796/1611] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:38:44 2022] POST /identity/v3/projects => generated 403 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:44.763741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308f333-6059-406c-a840-c0148ede939f admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:44.773592 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b33472-d545-42a1-91fb-67748e4f17ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:44.775665 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b33472-d545-42a1-91fb-67748e4f17ba service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:44.789976 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 816/1612] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:38:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:38:44.791675 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdccf52-89e5-4f60-af07-4f2e356fd202 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:44.792194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdccf52-89e5-4f60-af07-4f2e356fd202 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:44.792643 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdccf52-89e5-4f60-af07-4f2e356fd202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:44.830799 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-0bdccf52-89e5-4f60-af07-4f2e356fd202 None None] MFA Rules not processed for user `fa8866699789473ca655509b6c3e6fe6`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:44.832049 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0bdccf52-89e5-4f60-af07-4f2e356fd202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:44.852993 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308f333-6059-406c-a840-c0148ede939f admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:44.861307 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da2feb-4681-4ff0-8b3c-45a6314dc443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:44.862134 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da2feb-4681-4ff0-8b3c-45a6314dc443 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:44.862485 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da2feb-4681-4ff0-8b3c-45a6314dc443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:44.863086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da2feb-4681-4ff0-8b3c-45a6314dc443 admin admin] PATH_INFO: `/v3/projects/c64a8e4fb04d43428de8807aa1f6b228/users/8b0ec9e9234f482e98cc15e617f181f7/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:44.887300 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da2feb-4681-4ff0-8b3c-45a6314dc443 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c64a8e4fb04d43428de8807aa1f6b228, user_id=8b0ec9e9234f482e98cc15e617f181f7, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:44.892770 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da2feb-4681-4ff0-8b3c-45a6314dc443 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:44.913740 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 797/1613] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:38:44 2022] PUT /identity/v3/projects/c64a8e4fb04d43428de8807aa1f6b228/users/8b0ec9e9234f482e98cc15e617f181f7/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:38:44.920248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cff415-11f3-45dc-981c-a6d8944bd459 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:44.920248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cff415-11f3-45dc-981c-a6d8944bd459 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:44.920248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cff415-11f3-45dc-981c-a6d8944bd459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:44.921329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 817/1614] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:44.922900 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdccf52-89e5-4f60-af07-4f2e356fd202 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:44.981108 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-82cff415-11f3-45dc-981c-a6d8944bd459 None None] MFA Rules not processed for user `dfe222e6e5f84b70845df382889a14f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:44.982478 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-82cff415-11f3-45dc-981c-a6d8944bd459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:44.982892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:45.037076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdccf52-89e5-4f60-af07-4f2e356fd202 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:45.069957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 798/1615] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:44 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:45.070947 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cff415-11f3-45dc-981c-a6d8944bd459 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:45.072557 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-82cff415-11f3-45dc-981c-a6d8944bd459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:45.129766 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703a407-8543-4b24-93a5-019155335feb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:45.130408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703a407-8543-4b24-93a5-019155335feb service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:45.130876 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703a407-8543-4b24-93a5-019155335feb service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:45.131163 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703a407-8543-4b24-93a5-019155335feb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:45.156535 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cff415-11f3-45dc-981c-a6d8944bd459 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:45.160617 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a02bd-5229-4365-a36c-f29d5b3b696f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:45.161267 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a02bd-5229-4365-a36c-f29d5b3b696f service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:45.161570 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a02bd-5229-4365-a36c-f29d5b3b696f service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:45.161846 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a02bd-5229-4365-a36c-f29d5b3b696f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:45.212916 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:45.235159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703a407-8543-4b24-93a5-019155335feb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:45.236618 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703a407-8543-4b24-93a5-019155335feb service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:45.249654 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 818/1616] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:38:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:38:45.250605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703a407-8543-4b24-93a5-019155335feb service nova] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:45.300902 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a02bd-5229-4365-a36c-f29d5b3b696f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:45.304081 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a02bd-5229-4365-a36c-f29d5b3b696f service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:45.333011 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 799/1617] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:38:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:38:45.334613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b15b88-3f2c-41fb-b836-4ed592aa4bc2 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:45.338177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b15b88-3f2c-41fb-b836-4ed592aa4bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:45.339563 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b15b88-3f2c-41fb-b836-4ed592aa4bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:45.356332 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703a407-8543-4b24-93a5-019155335feb service nova] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:45.368824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc84c36-4612-4f21-b3a4-73239909ef3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:45.370575 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc84c36-4612-4f21-b3a4-73239909ef3a admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:45.370870 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc84c36-4612-4f21-b3a4-73239909ef3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:45.371146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc84c36-4612-4f21-b3a4-73239909ef3a admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:45.373283 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc84c36-4612-4f21-b3a4-73239909ef3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:45.380232 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc84c36-4612-4f21-b3a4-73239909ef3a admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:45.410899 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-34b15b88-3f2c-41fb-b836-4ed592aa4bc2 None None] MFA Rules not processed for user `8b0ec9e9234f482e98cc15e617f181f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:45.412412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-34b15b88-3f2c-41fb-b836-4ed592aa4bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:45.421277 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:45.433384 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 819/1618] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:38:45 2022] POST /identity/v3/users => generated 308 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:45.438151 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52bd8f-5563-40f9-abae-25bff92722f9 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:45.438869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52bd8f-5563-40f9-abae-25bff92722f9 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:45.439177 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52bd8f-5563-40f9-abae-25bff92722f9 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:45.446683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 820/1619] 10.222.0.47 () {54 vars in 962 bytes} [Thu Aug 4 04:38:45 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:38:45.447868 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52bd8f-5563-40f9-abae-25bff92722f9 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:45.464918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52bd8f-5563-40f9-abae-25bff92722f9 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:45.474197 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f105505-07ff-47e6-9dcb-2a78aa83b304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:45.474967 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f105505-07ff-47e6-9dcb-2a78aa83b304 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:45.475253 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f105505-07ff-47e6-9dcb-2a78aa83b304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:45.476150 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f105505-07ff-47e6-9dcb-2a78aa83b304 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:45.476856 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f105505-07ff-47e6-9dcb-2a78aa83b304 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:45.480150 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f105505-07ff-47e6-9dcb-2a78aa83b304 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:45.552985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 821/1620] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:38:45 2022] POST /identity/v3/projects => generated 355 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:45.555036 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f105505-07ff-47e6-9dcb-2a78aa83b304 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:45.628041 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 800/1621] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:45.632314 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267f74b-686c-468c-a3a4-bdcf7ae6a894 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:45.632706 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267f74b-686c-468c-a3a4-bdcf7ae6a894 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:45.632963 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267f74b-686c-468c-a3a4-bdcf7ae6a894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:45.686651 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-4267f74b-686c-468c-a3a4-bdcf7ae6a894 None None] MFA Rules not processed for user `caa2901451aa4554bdfcac1bddbc620a`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:45.693190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-4267f74b-686c-468c-a3a4-bdcf7ae6a894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:45.717986 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f105505-07ff-47e6-9dcb-2a78aa83b304 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:45.732496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f8d10-a844-4319-bd3e-2bf6244cfe34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:45.733333 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f8d10-a844-4319-bd3e-2bf6244cfe34 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:45.733643 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f8d10-a844-4319-bd3e-2bf6244cfe34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:45.734477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f8d10-a844-4319-bd3e-2bf6244cfe34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:45.738533 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f8d10-a844-4319-bd3e-2bf6244cfe34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:45.741651 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f8d10-a844-4319-bd3e-2bf6244cfe34 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:45.771040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 822/1622] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:45.771688 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f8d10-a844-4319-bd3e-2bf6244cfe34 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:45.794573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f8d10-a844-4319-bd3e-2bf6244cfe34 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:45.800857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f5d1f-f094-46d8-a884-6b869808a212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:45.801105 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f5d1f-f094-46d8-a884-6b869808a212 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:45.801351 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f5d1f-f094-46d8-a884-6b869808a212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:45.801790 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f5d1f-f094-46d8-a884-6b869808a212 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:45.803801 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f5d1f-f094-46d8-a884-6b869808a212 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:45.805151 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f5d1f-f094-46d8-a884-6b869808a212 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:45.859452 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 823/1623] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:38:45 2022] POST /identity/v3/users => generated 284 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:45.859452 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f5d1f-f094-46d8-a884-6b869808a212 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:45.891360 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 801/1624] 10.222.0.47 () {58 vars in 1059 bytes} [Thu Aug 4 04:38:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:45.901407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea49599-46d3-4867-98b8-7145700c7677 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:45.901731 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea49599-46d3-4867-98b8-7145700c7677 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:45.902086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea49599-46d3-4867-98b8-7145700c7677 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:45.906568 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 802/1625] 10.222.0.47 () {54 vars in 912 bytes} [Thu Aug 4 04:38:45 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:38:45.907450 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea49599-46d3-4867-98b8-7145700c7677 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:45.940281 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea49599-46d3-4867-98b8-7145700c7677 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:45.953114 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aeb2f2-ab5e-4163-a346-b0e21500665a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:45.953114 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aeb2f2-ab5e-4163-a346-b0e21500665a admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:45.953708 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aeb2f2-ab5e-4163-a346-b0e21500665a admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:45.953708 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aeb2f2-ab5e-4163-a346-b0e21500665a admin admin] PATH_INFO: `/v3/projects/41ae014c83fe40eb95bc533e9bb17b18/users/5a1335e9002b4de8ba05c26c457798c4/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:45.989643 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aeb2f2-ab5e-4163-a346-b0e21500665a admin admin] RBAC: Authorizing `identity:create_grant(project_id=41ae014c83fe40eb95bc533e9bb17b18, user_id=5a1335e9002b4de8ba05c26c457798c4, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:45.992366 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aeb2f2-ab5e-4163-a346-b0e21500665a admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:45.998482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f5d1f-f094-46d8-a884-6b869808a212 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:46.004083 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85558a0-5318-478d-aaa3-7c9540441585 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:46.004531 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85558a0-5318-478d-aaa3-7c9540441585 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:46.004814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85558a0-5318-478d-aaa3-7c9540441585 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:46.005089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85558a0-5318-478d-aaa3-7c9540441585 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:46.023455 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 803/1626] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:38:45 2022] PUT /identity/v3/projects/41ae014c83fe40eb95bc533e9bb17b18/users/5a1335e9002b4de8ba05c26c457798c4/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:38:46.023455 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aeb2f2-ab5e-4163-a346-b0e21500665a admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:46.070065 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:46.137582 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85558a0-5318-478d-aaa3-7c9540441585 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:46.139074 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85558a0-5318-478d-aaa3-7c9540441585 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:46.164804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aeb2f2-ab5e-4163-a346-b0e21500665a admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:46.167909 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 824/1627] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:38:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:38:46.168551 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85558a0-5318-478d-aaa3-7c9540441585 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:46.180501 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe0db2-3fcb-4b8c-a6df-87a8dd51bed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:46.181148 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe0db2-3fcb-4b8c-a6df-87a8dd51bed7 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:46.181433 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe0db2-3fcb-4b8c-a6df-87a8dd51bed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:46.181798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe0db2-3fcb-4b8c-a6df-87a8dd51bed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:46.182451 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe0db2-3fcb-4b8c-a6df-87a8dd51bed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:46.184309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe0db2-3fcb-4b8c-a6df-87a8dd51bed7 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:46.184465 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85558a0-5318-478d-aaa3-7c9540441585 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:46.191322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:46.217424 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 804/1628] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:46.217424 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a390b5-1e42-440d-be4b-fa367aa7b97a None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:46.219397 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a390b5-1e42-440d-be4b-fa367aa7b97a None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:46.219817 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a390b5-1e42-440d-be4b-fa367aa7b97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:46.242002 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-48a390b5-1e42-440d-be4b-fa367aa7b97a None None] MFA Rules not processed for user `caa2901451aa4554bdfcac1bddbc620a`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:46.243271 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-48a390b5-1e42-440d-be4b-fa367aa7b97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:46.347184 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73b3b9-b5c2-4e17-9500-169927095b60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:46.347855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73b3b9-b5c2-4e17-9500-169927095b60 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:46.348101 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73b3b9-b5c2-4e17-9500-169927095b60 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:46.348378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73b3b9-b5c2-4e17-9500-169927095b60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:46.365812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 805/1629] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Aug 4 04:38:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:46.380290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a390b5-1e42-440d-be4b-fa367aa7b97a None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:46.408291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a390b5-1e42-440d-be4b-fa367aa7b97a None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:46.423008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ec770-18c2-44e9-ab11-fe51486ad59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:46.423532 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ec770-18c2-44e9-ab11-fe51486ad59d admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:46.423532 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ec770-18c2-44e9-ab11-fe51486ad59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:46.423821 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ec770-18c2-44e9-ab11-fe51486ad59d admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:46.424537 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ec770-18c2-44e9-ab11-fe51486ad59d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:46.426949 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ec770-18c2-44e9-ab11-fe51486ad59d admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:46.448028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 806/1630] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:46.449102 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ec770-18c2-44e9-ab11-fe51486ad59d admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:46.465649 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ec770-18c2-44e9-ab11-fe51486ad59d admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:46.475450 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf2af5-1c3e-4208-89e7-8ced75f93d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:46.476270 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf2af5-1c3e-4208-89e7-8ced75f93d39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:46.476864 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf2af5-1c3e-4208-89e7-8ced75f93d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:46.477852 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf2af5-1c3e-4208-89e7-8ced75f93d39 admin admin] PATH_INFO: `/v3/projects/00207df8167a4333af912e59be49b9a6/users/1b8364515e224123905eb48442fc516f/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:46.502388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf2af5-1c3e-4208-89e7-8ced75f93d39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00207df8167a4333af912e59be49b9a6, user_id=1b8364515e224123905eb48442fc516f, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:46.504725 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73b3b9-b5c2-4e17-9500-169927095b60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:46.505433 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf2af5-1c3e-4208-89e7-8ced75f93d39 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:46.506372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73b3b9-b5c2-4e17-9500-169927095b60 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:46.527547 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 825/1631] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:38:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:38:46.531674 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 807/1632] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:38:46 2022] PUT /identity/v3/projects/00207df8167a4333af912e59be49b9a6/users/1b8364515e224123905eb48442fc516f/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:38:46.533097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf2af5-1c3e-4208-89e7-8ced75f93d39 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:46.543762 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73b3b9-b5c2-4e17-9500-169927095b60 service nova] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:46.639208 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf2af5-1c3e-4208-89e7-8ced75f93d39 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:46.645913 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cbcc3-ec0c-4708-bdb2-96fe00ae07ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:46.647400 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cbcc3-ec0c-4708-bdb2-96fe00ae07ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:46.648427 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cbcc3-ec0c-4708-bdb2-96fe00ae07ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:46.648427 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cbcc3-ec0c-4708-bdb2-96fe00ae07ae admin admin] PATH_INFO: `/v3/projects/41ae014c83fe40eb95bc533e9bb17b18/users/5a1335e9002b4de8ba05c26c457798c4/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:46.651818 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97cbcc3-ec0c-4708-bdb2-96fe00ae07ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=41ae014c83fe40eb95bc533e9bb17b18, user_id=5a1335e9002b4de8ba05c26c457798c4, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:46.656292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97cbcc3-ec0c-4708-bdb2-96fe00ae07ae admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:46.685057 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 808/1633] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:38:46 2022] PUT /identity/v3/projects/41ae014c83fe40eb95bc533e9bb17b18/users/5a1335e9002b4de8ba05c26c457798c4/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:38:46.686582 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cbcc3-ec0c-4708-bdb2-96fe00ae07ae admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:46.700766 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:46.735468 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73b3b9-b5c2-4e17-9500-169927095b60 service nova] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:46.784567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cbcc3-ec0c-4708-bdb2-96fe00ae07ae admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:46.790529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb112d4-62ed-4f87-875d-740123e41fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:46.791276 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb112d4-62ed-4f87-875d-740123e41fdb admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:46.791566 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb112d4-62ed-4f87-875d-740123e41fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:46.791839 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb112d4-62ed-4f87-875d-740123e41fdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:46.792613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb112d4-62ed-4f87-875d-740123e41fdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:46.796854 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb112d4-62ed-4f87-875d-740123e41fdb admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:46.816826 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 809/1634] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:46.819042 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8d082-a21a-4bbd-8300-aa96cd8e812f None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:46.819335 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8d082-a21a-4bbd-8300-aa96cd8e812f None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:46.819768 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8d082-a21a-4bbd-8300-aa96cd8e812f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:46.842256 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578928b-ad62-4dd9-8ea7-8315a10aa8fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:46.842896 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578928b-ad62-4dd9-8ea7-8315a10aa8fd service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:46.843716 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578928b-ad62-4dd9-8ea7-8315a10aa8fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:46.843993 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578928b-ad62-4dd9-8ea7-8315a10aa8fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:46.860965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-a5d8d082-a21a-4bbd-8300-aa96cd8e812f None None] MFA Rules not processed for user `5a1335e9002b4de8ba05c26c457798c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:46.862497 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-a5d8d082-a21a-4bbd-8300-aa96cd8e812f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:46.872645 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:46.985745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 810/1635] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:46 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:46.986742 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8d082-a21a-4bbd-8300-aa96cd8e812f None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:47.000477 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8d082-a21a-4bbd-8300-aa96cd8e812f None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:47.014339 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba7483-869f-498a-8bda-a6daf7ed2199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:47.015224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba7483-869f-498a-8bda-a6daf7ed2199 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:47.020143 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba7483-869f-498a-8bda-a6daf7ed2199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:47.021696 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba7483-869f-498a-8bda-a6daf7ed2199 admin admin] PATH_INFO: `/v3/projects/00207df8167a4333af912e59be49b9a6/users/1b8364515e224123905eb48442fc516f/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:47.023928 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578928b-ad62-4dd9-8ea7-8315a10aa8fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:47.026555 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba7483-869f-498a-8bda-a6daf7ed2199 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00207df8167a4333af912e59be49b9a6, user_id=1b8364515e224123905eb48442fc516f, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:47.030505 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578928b-ad62-4dd9-8ea7-8315a10aa8fd service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:47.036684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba7483-869f-498a-8bda-a6daf7ed2199 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:47.067400 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 811/1636] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:38:46 2022] PUT /identity/v3/projects/00207df8167a4333af912e59be49b9a6/users/1b8364515e224123905eb48442fc516f/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:38:47.080754 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b90ed2-9c3e-48b3-ac02-d685545b6451 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:47.081047 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b90ed2-9c3e-48b3-ac02-d685545b6451 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:47.081323 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b90ed2-9c3e-48b3-ac02-d685545b6451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:47.084820 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 826/1637] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:38:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:38:47.137808 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578928b-ad62-4dd9-8ea7-8315a10aa8fd service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:47.149794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578928b-ad62-4dd9-8ea7-8315a10aa8fd service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:47.190078 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-a6b90ed2-9c3e-48b3-ac02-d685545b6451 None None] MFA Rules not processed for user `1b8364515e224123905eb48442fc516f`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:47.192621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-a6b90ed2-9c3e-48b3-ac02-d685545b6451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:47.270490 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04688b-4a20-4e57-a6ac-fec33ffe1cd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:47.272254 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04688b-4a20-4e57-a6ac-fec33ffe1cd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:47.273269 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04688b-4a20-4e57-a6ac-fec33ffe1cd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:47.275095 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04688b-4a20-4e57-a6ac-fec33ffe1cd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:47.381883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:47.394318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 812/1638] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:47 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:47.396349 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddb6b1-87e6-4315-991a-417bad2d4b0a None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:47.398996 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddb6b1-87e6-4315-991a-417bad2d4b0a None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:47.399259 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddb6b1-87e6-4315-991a-417bad2d4b0a None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:47.403236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 813/1639] 10.222.0.47 () {54 vars in 912 bytes} [Thu Aug 4 04:38:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:38:47.407193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d15f29-82a2-4612-9eec-588f978182af None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:47.407582 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d15f29-82a2-4612-9eec-588f978182af None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:47.407951 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d15f29-82a2-4612-9eec-588f978182af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:47.454118 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-c6d15f29-82a2-4612-9eec-588f978182af None None] MFA Rules not processed for user `caa2901451aa4554bdfcac1bddbc620a`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:47.456443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be04688b-4a20-4e57-a6ac-fec33ffe1cd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:47.458623 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be04688b-4a20-4e57-a6ac-fec33ffe1cd3 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:47.460738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c6d15f29-82a2-4612-9eec-588f978182af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:47.497191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 827/1640] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:38:47 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:38:47.602883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 814/1641] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Aug 4 04:38:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:47.845322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04688b-4a20-4e57-a6ac-fec33ffe1cd3 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:47.859319 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04688b-4a20-4e57-a6ac-fec33ffe1cd3 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:47.866346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad37bc2-d6bc-4ffb-ab14-3aa5f48128cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:47.870293 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad37bc2-d6bc-4ffb-ab14-3aa5f48128cf service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:47.870669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad37bc2-d6bc-4ffb-ab14-3aa5f48128cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:47.870967 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad37bc2-d6bc-4ffb-ab14-3aa5f48128cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:47.896407 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad37bc2-d6bc-4ffb-ab14-3aa5f48128cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:47.898225 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad37bc2-d6bc-4ffb-ab14-3aa5f48128cf service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:47.932920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 828/1642] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:38:47 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:38:48.015773 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d15f29-82a2-4612-9eec-588f978182af None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:48.032920 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d15f29-82a2-4612-9eec-588f978182af None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:48.043123 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eaad0e-4600-4fd9-9d0e-ded3c8d9171d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:48.043123 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eaad0e-4600-4fd9-9d0e-ded3c8d9171d service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:48.043425 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eaad0e-4600-4fd9-9d0e-ded3c8d9171d service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:48.043425 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eaad0e-4600-4fd9-9d0e-ded3c8d9171d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:48.069787 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eaad0e-4600-4fd9-9d0e-ded3c8d9171d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:48.072575 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eaad0e-4600-4fd9-9d0e-ded3c8d9171d service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:48.095193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 815/1643] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:38:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:38:48.383445 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad37bc2-d6bc-4ffb-ab14-3aa5f48128cf service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:48.402185 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad37bc2-d6bc-4ffb-ab14-3aa5f48128cf service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:48.416624 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3f475-3025-4861-858f-923ff4a1870c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:48.423676 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3f475-3025-4861-858f-923ff4a1870c service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:48.424253 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3f475-3025-4861-858f-923ff4a1870c service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:48.424666 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3f475-3025-4861-858f-923ff4a1870c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:48.529691 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:48.574647 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3f475-3025-4861-858f-923ff4a1870c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:48.576205 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3f475-3025-4861-858f-923ff4a1870c service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:48.603318 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 829/1644] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:38:48 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:38:48.828329 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eaad0e-4600-4fd9-9d0e-ded3c8d9171d service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:49.064030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eaad0e-4600-4fd9-9d0e-ded3c8d9171d service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:49.076486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac236621-3e0c-41b3-b038-8270e0ebad44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:49.077175 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac236621-3e0c-41b3-b038-8270e0ebad44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:49.077525 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac236621-3e0c-41b3-b038-8270e0ebad44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:49.077876 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac236621-3e0c-41b3-b038-8270e0ebad44 admin admin] PATH_INFO: `/v3/users/714e4bd548f144959cdd903d226d7c3b` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:49.083728 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac236621-3e0c-41b3-b038-8270e0ebad44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=714e4bd548f144959cdd903d226d7c3b)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:49.085970 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac236621-3e0c-41b3-b038-8270e0ebad44 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:49.297519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-ac236621-3e0c-41b3-b038-8270e0ebad44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '714e4bd548f144959cdd903d226d7c3b'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:49.298871 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-ac236621-3e0c-41b3-b038-8270e0ebad44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '714e4bd548f144959cdd903d226d7c3b'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:49.329276 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-ac236621-3e0c-41b3-b038-8270e0ebad44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '714e4bd548f144959cdd903d226d7c3b'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:49.345994 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-ac236621-3e0c-41b3-b038-8270e0ebad44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '714e4bd548f144959cdd903d226d7c3b'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:49.347135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-ac236621-3e0c-41b3-b038-8270e0ebad44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '714e4bd548f144959cdd903d226d7c3b'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:49.369888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:49.387027 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 816/1645] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:38:48 2022] DELETE /identity/v3/users/714e4bd548f144959cdd903d226d7c3b => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:38:49.393232 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c54673-fb93-43ca-8cb6-afc07772997b None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:49.393456 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c54673-fb93-43ca-8cb6-afc07772997b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:49.393667 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c54673-fb93-43ca-8cb6-afc07772997b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:49.443759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b9c54673-fb93-43ca-8cb6-afc07772997b None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:49.448575 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b9c54673-fb93-43ca-8cb6-afc07772997b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:49.630098 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 830/1646] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:49.736111 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac236621-3e0c-41b3-b038-8270e0ebad44 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:49.737697 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-ac236621-3e0c-41b3-b038-8270e0ebad44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:49.911722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac236621-3e0c-41b3-b038-8270e0ebad44 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:49.926812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c897a2-0025-4ccb-934c-913ac9f208d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:49.927536 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c897a2-0025-4ccb-934c-913ac9f208d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:49.928704 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c897a2-0025-4ccb-934c-913ac9f208d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:49.929622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c897a2-0025-4ccb-934c-913ac9f208d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:49.958350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c54673-fb93-43ca-8cb6-afc07772997b None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:49.958350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b9c54673-fb93-43ca-8cb6-afc07772997b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:49.981849 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c897a2-0025-4ccb-934c-913ac9f208d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:49.989255 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c897a2-0025-4ccb-934c-913ac9f208d6 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:49.995550 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c54673-fb93-43ca-8cb6-afc07772997b None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:50.013253 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d4ab4-9f21-47d6-a74b-38094d49979e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:50.013955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d4ab4-9f21-47d6-a74b-38094d49979e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:50.014239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d4ab4-9f21-47d6-a74b-38094d49979e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:50.014511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d4ab4-9f21-47d6-a74b-38094d49979e admin admin] PATH_INFO: `/v3/users/85b8ce0f906e4f1796ca66fb4b98fc2b` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:50.023003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d4ab4-9f21-47d6-a74b-38094d49979e admin admin] RBAC: Authorizing `identity:delete_user(user_id=85b8ce0f906e4f1796ca66fb4b98fc2b)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:50.029359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: WARNING py.warnings [None req-b06d4ab4-9f21-47d6-a74b-38094d49979e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 04:38:50.029359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: warnings.warn(msg) Aug 04 04:38:50.029359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: Aug 04 04:38:50.032710 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d4ab4-9f21-47d6-a74b-38094d49979e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:50.033709 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 817/1647] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:38:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:38:50.310683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b06d4ab4-9f21-47d6-a74b-38094d49979e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85b8ce0f906e4f1796ca66fb4b98fc2b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:50.385946 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b06d4ab4-9f21-47d6-a74b-38094d49979e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85b8ce0f906e4f1796ca66fb4b98fc2b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:50.385946 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b06d4ab4-9f21-47d6-a74b-38094d49979e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85b8ce0f906e4f1796ca66fb4b98fc2b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:50.467153 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b06d4ab4-9f21-47d6-a74b-38094d49979e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85b8ce0f906e4f1796ca66fb4b98fc2b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:50.469135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b06d4ab4-9f21-47d6-a74b-38094d49979e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85b8ce0f906e4f1796ca66fb4b98fc2b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:50.519899 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a90505-1036-4112-8824-fecb620de032 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:50.520215 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a90505-1036-4112-8824-fecb620de032 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:50.520504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a90505-1036-4112-8824-fecb620de032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:50.520869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 831/1648] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:38:49 2022] DELETE /identity/v3/users/85b8ce0f906e4f1796ca66fb4b98fc2b => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:38:50.583542 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-e1a90505-1036-4112-8824-fecb620de032 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:50.589247 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-e1a90505-1036-4112-8824-fecb620de032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:50.734879 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 818/1649] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:50.814215 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d4ab4-9f21-47d6-a74b-38094d49979e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:50.823909 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b06d4ab4-9f21-47d6-a74b-38094d49979e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:51.068616 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d4ab4-9f21-47d6-a74b-38094d49979e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:51.082957 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7ad24-5fea-47bf-8b83-5dd9e331c874 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:51.086422 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7ad24-5fea-47bf-8b83-5dd9e331c874 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:51.087156 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7ad24-5fea-47bf-8b83-5dd9e331c874 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:51.088730 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7ad24-5fea-47bf-8b83-5dd9e331c874 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:51.126902 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:51.138061 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7ad24-5fea-47bf-8b83-5dd9e331c874 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:51.141649 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7ad24-5fea-47bf-8b83-5dd9e331c874 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:51.174473 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 832/1650] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:38:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:38:51.718796 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a90505-1036-4112-8824-fecb620de032 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:51.721223 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-e1a90505-1036-4112-8824-fecb620de032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:51.743651 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a90505-1036-4112-8824-fecb620de032 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:51.759385 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423b19b-7039-4534-9c6b-3fb7dd7e367b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:51.764367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423b19b-7039-4534-9c6b-3fb7dd7e367b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:51.767257 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423b19b-7039-4534-9c6b-3fb7dd7e367b admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:51.769418 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423b19b-7039-4534-9c6b-3fb7dd7e367b admin admin] PATH_INFO: `/v3/projects/f3b4625caa104a7ebb014b7fa76494a3` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:51.774079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5423b19b-7039-4534-9c6b-3fb7dd7e367b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3b4625caa104a7ebb014b7fa76494a3)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:51.777118 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: WARNING py.warnings [None req-5423b19b-7039-4534-9c6b-3fb7dd7e367b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 04:38:51.777118 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: warnings.warn(msg) Aug 04 04:38:51.777118 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: Aug 04 04:38:51.782249 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5423b19b-7039-4534-9c6b-3fb7dd7e367b admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:52.015363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5423b19b-7039-4534-9c6b-3fb7dd7e367b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3b4625caa104a7ebb014b7fa76494a3', 'request_id': 'req-5423b19b-7039-4534-9c6b-3fb7dd7e367b'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:52.048028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5423b19b-7039-4534-9c6b-3fb7dd7e367b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3b4625caa104a7ebb014b7fa76494a3', 'request_id': 'req-5423b19b-7039-4534-9c6b-3fb7dd7e367b'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:52.060630 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5423b19b-7039-4534-9c6b-3fb7dd7e367b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3b4625caa104a7ebb014b7fa76494a3', 'request_id': 'req-5423b19b-7039-4534-9c6b-3fb7dd7e367b'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:52.127032 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5423b19b-7039-4534-9c6b-3fb7dd7e367b admin admin] The token cache is being invalidate because project f3b4625caa104a7ebb014b7fa76494a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:38:52.129051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5423b19b-7039-4534-9c6b-3fb7dd7e367b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:52.131709 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5423b19b-7039-4534-9c6b-3fb7dd7e367b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:52.133781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 819/1651] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:38:51 2022] DELETE /identity/v3/projects/f3b4625caa104a7ebb014b7fa76494a3 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:38:52.138255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7ad24-5fea-47bf-8b83-5dd9e331c874 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:52.153629 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7ad24-5fea-47bf-8b83-5dd9e331c874 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:52.353609 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6318f79b-79aa-4360-a791-16b6af1aea11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:52.354808 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318f79b-79aa-4360-a791-16b6af1aea11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:52.356150 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318f79b-79aa-4360-a791-16b6af1aea11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:52.361071 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318f79b-79aa-4360-a791-16b6af1aea11 admin admin] PATH_INFO: `/v3/users/5a1335e9002b4de8ba05c26c457798c4` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:52.364417 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6318f79b-79aa-4360-a791-16b6af1aea11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a1335e9002b4de8ba05c26c457798c4)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:52.372979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6318f79b-79aa-4360-a791-16b6af1aea11 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:52.538311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6318f79b-79aa-4360-a791-16b6af1aea11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a1335e9002b4de8ba05c26c457798c4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:52.557223 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6318f79b-79aa-4360-a791-16b6af1aea11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a1335e9002b4de8ba05c26c457798c4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:52.558861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6318f79b-79aa-4360-a791-16b6af1aea11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a1335e9002b4de8ba05c26c457798c4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:52.644716 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6318f79b-79aa-4360-a791-16b6af1aea11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a1335e9002b4de8ba05c26c457798c4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:52.646103 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6318f79b-79aa-4360-a791-16b6af1aea11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a1335e9002b4de8ba05c26c457798c4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:52.707329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 833/1652] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:38:52 2022] DELETE /identity/v3/users/5a1335e9002b4de8ba05c26c457798c4 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:38:53.013291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423b19b-7039-4534-9c6b-3fb7dd7e367b admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:53.016695 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5423b19b-7039-4534-9c6b-3fb7dd7e367b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:53.042189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423b19b-7039-4534-9c6b-3fb7dd7e367b admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:53.131910 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:53.253625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713eed2-099b-4120-9556-66116de35f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:53.255615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713eed2-099b-4120-9556-66116de35f19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:53.256009 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713eed2-099b-4120-9556-66116de35f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:53.256402 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713eed2-099b-4120-9556-66116de35f19 admin admin] PATH_INFO: `/v3/projects/26ffb724e028422eb6f69be51973fe9b` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:53.260030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5713eed2-099b-4120-9556-66116de35f19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26ffb724e028422eb6f69be51973fe9b)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:53.265802 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5713eed2-099b-4120-9556-66116de35f19 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:53.410552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5713eed2-099b-4120-9556-66116de35f19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26ffb724e028422eb6f69be51973fe9b', 'request_id': 'req-5713eed2-099b-4120-9556-66116de35f19'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:53.420779 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5713eed2-099b-4120-9556-66116de35f19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26ffb724e028422eb6f69be51973fe9b', 'request_id': 'req-5713eed2-099b-4120-9556-66116de35f19'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:53.432885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5713eed2-099b-4120-9556-66116de35f19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26ffb724e028422eb6f69be51973fe9b', 'request_id': 'req-5713eed2-099b-4120-9556-66116de35f19'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:53.448095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5713eed2-099b-4120-9556-66116de35f19 admin admin] The token cache is being invalidate because project 26ffb724e028422eb6f69be51973fe9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:38:53.448801 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5713eed2-099b-4120-9556-66116de35f19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:53.449762 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5713eed2-099b-4120-9556-66116de35f19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:53.451590 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 820/1653] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:38:53 2022] DELETE /identity/v3/projects/26ffb724e028422eb6f69be51973fe9b => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:38:53.456904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d270d4d-8de4-4392-8e0f-e44bd3d14339 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:53.458124 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d270d4d-8de4-4392-8e0f-e44bd3d14339 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:53.458633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d270d4d-8de4-4392-8e0f-e44bd3d14339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:53.487882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-4d270d4d-8de4-4392-8e0f-e44bd3d14339 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:53.489460 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4d270d4d-8de4-4392-8e0f-e44bd3d14339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:53.587606 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bda4b-d8b2-47e0-9dc9-8a2ca1c7a9dd None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:53.588112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bda4b-d8b2-47e0-9dc9-8a2ca1c7a9dd None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:53.588545 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bda4b-d8b2-47e0-9dc9-8a2ca1c7a9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:53.591825 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 834/1654] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:53.616420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-567bda4b-d8b2-47e0-9dc9-8a2ca1c7a9dd None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:53.618867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-567bda4b-d8b2-47e0-9dc9-8a2ca1c7a9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:53.636418 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d270d4d-8de4-4392-8e0f-e44bd3d14339 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:53.646968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 821/1655] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:53.649963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d270d4d-8de4-4392-8e0f-e44bd3d14339 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:53.678413 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1b920-d86b-493b-bea8-121577479cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:53.679069 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1b920-d86b-493b-bea8-121577479cdd admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:53.679407 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1b920-d86b-493b-bea8-121577479cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:53.679731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1b920-d86b-493b-bea8-121577479cdd admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:53.680533 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1b920-d86b-493b-bea8-121577479cdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:38:53.680849 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1b920-d86b-493b-bea8-121577479cdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:53.682518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1b920-d86b-493b-bea8-121577479cdd admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:53.703936 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:53.714546 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bda4b-d8b2-47e0-9dc9-8a2ca1c7a9dd None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:53.727238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 835/1656] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:38:53.727579 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bda4b-d8b2-47e0-9dc9-8a2ca1c7a9dd None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:53.739719 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5135962-7acb-4125-866f-b9c33f787eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:53.740377 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5135962-7acb-4125-866f-b9c33f787eb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:53.740657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5135962-7acb-4125-866f-b9c33f787eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:53.740942 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5135962-7acb-4125-866f-b9c33f787eb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:53.742334 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5135962-7acb-4125-866f-b9c33f787eb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:53.747568 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5135962-7acb-4125-866f-b9c33f787eb0 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:53.828229 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1b920-d86b-493b-bea8-121577479cdd admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:53.838075 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 822/1657] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:38:53 2022] POST /identity/v3/projects => generated 405 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:53.846473 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1b920-d86b-493b-bea8-121577479cdd admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:53.960806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2288dd-45bd-4f67-bd6f-ca08fa387357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:53.961681 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2288dd-45bd-4f67-bd6f-ca08fa387357 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:53.962618 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2288dd-45bd-4f67-bd6f-ca08fa387357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:53.962932 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2288dd-45bd-4f67-bd6f-ca08fa387357 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:53.966750 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2288dd-45bd-4f67-bd6f-ca08fa387357 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:53.968842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2288dd-45bd-4f67-bd6f-ca08fa387357 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:54.008256 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5135962-7acb-4125-866f-b9c33f787eb0 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:54.027180 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 836/1658] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:38:53 2022] POST /identity/v3/users => generated 303 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 04:38:54.029361 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2288dd-45bd-4f67-bd6f-ca08fa387357 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:54.033362 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9e2288dd-45bd-4f67-bd6f-ca08fa387357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:54.039525 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5135962-7acb-4125-866f-b9c33f787eb0 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:54.058578 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417653d-4f21-4dab-9d4a-54fd51435c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:54.058723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417653d-4f21-4dab-9d4a-54fd51435c42 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:54.058991 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417653d-4f21-4dab-9d4a-54fd51435c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:54.059265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417653d-4f21-4dab-9d4a-54fd51435c42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:54.059982 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c417653d-4f21-4dab-9d4a-54fd51435c42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:54.061701 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c417653d-4f21-4dab-9d4a-54fd51435c42 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:54.066894 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2288dd-45bd-4f67-bd6f-ca08fa387357 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:54.080552 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41485b3-d397-457d-aae1-449bb06cac05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:54.081277 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41485b3-d397-457d-aae1-449bb06cac05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:54.081576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41485b3-d397-457d-aae1-449bb06cac05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:54.081858 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41485b3-d397-457d-aae1-449bb06cac05 admin admin] PATH_INFO: `/v3/projects/41ae014c83fe40eb95bc533e9bb17b18` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:54.085068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41485b3-d397-457d-aae1-449bb06cac05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41ae014c83fe40eb95bc533e9bb17b18)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:54.091028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41485b3-d397-457d-aae1-449bb06cac05 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:54.096463 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 823/1659] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 04:38:54.103806 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417653d-4f21-4dab-9d4a-54fd51435c42 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:54.140550 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417653d-4f21-4dab-9d4a-54fd51435c42 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:54.154172 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e59826-6c0f-47e8-a334-8d728717c019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:54.154825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e59826-6c0f-47e8-a334-8d728717c019 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:54.155107 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e59826-6c0f-47e8-a334-8d728717c019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:54.155952 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e59826-6c0f-47e8-a334-8d728717c019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:54.156893 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e59826-6c0f-47e8-a334-8d728717c019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:54.165210 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e59826-6c0f-47e8-a334-8d728717c019 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:54.206407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 824/1660] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:54.209353 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e59826-6c0f-47e8-a334-8d728717c019 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:54.237240 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e59826-6c0f-47e8-a334-8d728717c019 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:54.240777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a41485b3-d397-457d-aae1-449bb06cac05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41ae014c83fe40eb95bc533e9bb17b18', 'request_id': 'req-a41485b3-d397-457d-aae1-449bb06cac05'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:54.259883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a41485b3-d397-457d-aae1-449bb06cac05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41ae014c83fe40eb95bc533e9bb17b18', 'request_id': 'req-a41485b3-d397-457d-aae1-449bb06cac05'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:54.271908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a41485b3-d397-457d-aae1-449bb06cac05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41ae014c83fe40eb95bc533e9bb17b18', 'request_id': 'req-a41485b3-d397-457d-aae1-449bb06cac05'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:54.306091 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a41485b3-d397-457d-aae1-449bb06cac05 admin admin] The token cache is being invalidate because project 41ae014c83fe40eb95bc533e9bb17b18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:38:54.309638 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a41485b3-d397-457d-aae1-449bb06cac05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:54.311074 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a41485b3-d397-457d-aae1-449bb06cac05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:54.316218 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 837/1661] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:38:54 2022] DELETE /identity/v3/projects/41ae014c83fe40eb95bc533e9bb17b18 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:38:54.322990 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621c9ea-2539-49dc-9548-84441252da3b None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:54.323322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621c9ea-2539-49dc-9548-84441252da3b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:54.323611 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621c9ea-2539-49dc-9548-84441252da3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:54.399498 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-3621c9ea-2539-49dc-9548-84441252da3b None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:54.404099 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3621c9ea-2539-49dc-9548-84441252da3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:54.433610 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79485f7-e6a8-4c39-8bfe-86e1a899c3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:54.434989 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79485f7-e6a8-4c39-8bfe-86e1a899c3cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:54.435303 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79485f7-e6a8-4c39-8bfe-86e1a899c3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:54.435632 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79485f7-e6a8-4c39-8bfe-86e1a899c3cf admin admin] PATH_INFO: `/v3/projects/7598ff62196641f39835ee1b894f151e/users/b3acc8bc894e4541bf529a7cd104c958/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:54.456352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:54.479056 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79485f7-e6a8-4c39-8bfe-86e1a899c3cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7598ff62196641f39835ee1b894f151e, user_id=b3acc8bc894e4541bf529a7cd104c958, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:54.486820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79485f7-e6a8-4c39-8bfe-86e1a899c3cf admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:54.527241 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 825/1662] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:38:54 2022] PUT /identity/v3/projects/7598ff62196641f39835ee1b894f151e/users/b3acc8bc894e4541bf529a7cd104c958/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:38:54.533154 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad14db-e38e-4dad-8f6e-a3d4170f8a20 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:54.533481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad14db-e38e-4dad-8f6e-a3d4170f8a20 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:54.533764 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad14db-e38e-4dad-8f6e-a3d4170f8a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:54.563761 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-fcad14db-e38e-4dad-8f6e-a3d4170f8a20 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:54.565813 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-fcad14db-e38e-4dad-8f6e-a3d4170f8a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:54.567150 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 838/1663] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:54.575278 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda4f6f-a8b8-4c0c-a965-b326023305c1 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:54.575998 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda4f6f-a8b8-4c0c-a965-b326023305c1 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:54.577502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda4f6f-a8b8-4c0c-a965-b326023305c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:54.628468 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-2bda4f6f-a8b8-4c0c-a965-b326023305c1 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:54.631722 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2bda4f6f-a8b8-4c0c-a965-b326023305c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:54.648960 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:54.685679 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 826/1664] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:54.739969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad14db-e38e-4dad-8f6e-a3d4170f8a20 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:54.742555 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-fcad14db-e38e-4dad-8f6e-a3d4170f8a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:54.767499 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 839/1665] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:38:54.774454 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda4f6f-a8b8-4c0c-a965-b326023305c1 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:54.802740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda4f6f-a8b8-4c0c-a965-b326023305c1 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:54.816534 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c7c67-0a63-47b5-b0a9-a72c610c24bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:54.818582 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c7c67-0a63-47b5-b0a9-a72c610c24bc admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:54.819474 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c7c67-0a63-47b5-b0a9-a72c610c24bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:54.820394 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c7c67-0a63-47b5-b0a9-a72c610c24bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:54.823413 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391c7c67-0a63-47b5-b0a9-a72c610c24bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:38:54.824123 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391c7c67-0a63-47b5-b0a9-a72c610c24bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:54.826686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391c7c67-0a63-47b5-b0a9-a72c610c24bc admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:54.860625 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 840/1666] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:38:54.864615 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c7c67-0a63-47b5-b0a9-a72c610c24bc admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:54.898170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c7c67-0a63-47b5-b0a9-a72c610c24bc admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:54.910386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd7af9-f9e3-4e01-9bce-29e6c5efb402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:54.911354 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd7af9-f9e3-4e01-9bce-29e6c5efb402 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:54.912674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd7af9-f9e3-4e01-9bce-29e6c5efb402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:54.914493 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd7af9-f9e3-4e01-9bce-29e6c5efb402 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:54.915623 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd7af9-f9e3-4e01-9bce-29e6c5efb402 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:54.940208 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd7af9-f9e3-4e01-9bce-29e6c5efb402 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:54.970031 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad14db-e38e-4dad-8f6e-a3d4170f8a20 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:54.991337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf91458-40c8-4c0a-af2e-06dd123548e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:54.992038 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf91458-40c8-4c0a-af2e-06dd123548e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:54.992527 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf91458-40c8-4c0a-af2e-06dd123548e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:54.993368 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf91458-40c8-4c0a-af2e-06dd123548e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:55.032265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf91458-40c8-4c0a-af2e-06dd123548e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:55.033692 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf91458-40c8-4c0a-af2e-06dd123548e5 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:55.063203 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 827/1667] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:38:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:38:55.087325 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 841/1668] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:38:54 2022] POST /identity/v3/projects => generated 367 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:55.091624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf91458-40c8-4c0a-af2e-06dd123548e5 service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:55.113204 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf91458-40c8-4c0a-af2e-06dd123548e5 service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:55.283836 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d4b6d-ccdd-48bc-962f-2459392a5c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:55.284493 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d4b6d-ccdd-48bc-962f-2459392a5c64 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:55.284770 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d4b6d-ccdd-48bc-962f-2459392a5c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:55.285039 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d4b6d-ccdd-48bc-962f-2459392a5c64 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:55.285777 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d4b6d-ccdd-48bc-962f-2459392a5c64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:55.288224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d4b6d-ccdd-48bc-962f-2459392a5c64 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:55.348922 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 828/1669] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:38:55 2022] POST /identity/v3/users => generated 284 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:55.350082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd7af9-f9e3-4e01-9bce-29e6c5efb402 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:55.370597 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd7af9-f9e3-4e01-9bce-29e6c5efb402 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:55.382182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1ba8d-e37a-4317-8cb7-02c3aebc19dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:55.383032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1ba8d-e37a-4317-8cb7-02c3aebc19dc admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:55.383490 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1ba8d-e37a-4317-8cb7-02c3aebc19dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:55.383883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1ba8d-e37a-4317-8cb7-02c3aebc19dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:55.384676 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1ba8d-e37a-4317-8cb7-02c3aebc19dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:55.386555 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1ba8d-e37a-4317-8cb7-02c3aebc19dc admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:55.404397 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 842/1670] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:55.407639 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d4b6d-ccdd-48bc-962f-2459392a5c64 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:55.418745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d4b6d-ccdd-48bc-962f-2459392a5c64 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:55.427907 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb26784-49c3-4a22-9af4-b981394fe884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:55.428878 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb26784-49c3-4a22-9af4-b981394fe884 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:55.429326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb26784-49c3-4a22-9af4-b981394fe884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:55.429958 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb26784-49c3-4a22-9af4-b981394fe884 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:55.431765 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb26784-49c3-4a22-9af4-b981394fe884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:55.434155 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb26784-49c3-4a22-9af4-b981394fe884 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:55.461615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 829/1671] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:55.463351 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1ba8d-e37a-4317-8cb7-02c3aebc19dc admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:55.488497 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1ba8d-e37a-4317-8cb7-02c3aebc19dc admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:55.495927 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc582dce-cbd6-4544-aae9-7a21bb0372e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:55.496754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc582dce-cbd6-4544-aae9-7a21bb0372e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:55.497212 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc582dce-cbd6-4544-aae9-7a21bb0372e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:55.497662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc582dce-cbd6-4544-aae9-7a21bb0372e7 admin admin] PATH_INFO: `/v3/projects/12042b3fe2fa46b8bc1cbc34728948cd/users/fc33224cc8164ccab1859c97cb9c24e7/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:55.546290 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc582dce-cbd6-4544-aae9-7a21bb0372e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12042b3fe2fa46b8bc1cbc34728948cd, user_id=fc33224cc8164ccab1859c97cb9c24e7, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:55.549217 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc582dce-cbd6-4544-aae9-7a21bb0372e7 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:55.569171 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 843/1672] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:38:55 2022] PUT /identity/v3/projects/12042b3fe2fa46b8bc1cbc34728948cd/users/fc33224cc8164ccab1859c97cb9c24e7/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:38:55.583040 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f885a-731f-439a-87a8-382e42a1f16e None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:55.583397 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f885a-731f-439a-87a8-382e42a1f16e None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:55.583819 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f885a-731f-439a-87a8-382e42a1f16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:55.623385 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:55.651287 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-425f885a-731f-439a-87a8-382e42a1f16e None None] MFA Rules not processed for user `fc33224cc8164ccab1859c97cb9c24e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:55.653838 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-425f885a-731f-439a-87a8-382e42a1f16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:55.801995 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd9de6-f976-4eb1-8cfc-bfd0fe543688 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:55.802630 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd9de6-f976-4eb1-8cfc-bfd0fe543688 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:55.802992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd9de6-f976-4eb1-8cfc-bfd0fe543688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:55.893737 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-63dd9de6-f976-4eb1-8cfc-bfd0fe543688 None None] MFA Rules not processed for user `99d00c09d8374f59ac1260ade95be9ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:55.906126 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-63dd9de6-f976-4eb1-8cfc-bfd0fe543688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:55.927745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 830/1673] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:55.942403 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f885a-731f-439a-87a8-382e42a1f16e None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:56.085023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 844/1674] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:56.097807 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57ebe3-723c-4d67-b1ec-d6f66b2a4cb5 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:56.100222 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57ebe3-723c-4d67-b1ec-d6f66b2a4cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:56.100444 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57ebe3-723c-4d67-b1ec-d6f66b2a4cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:56.109230 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f885a-731f-439a-87a8-382e42a1f16e None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:56.131440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcb848-8bf5-4016-aef2-3ff5aaeeac68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:56.132146 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcb848-8bf5-4016-aef2-3ff5aaeeac68 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:56.132430 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcb848-8bf5-4016-aef2-3ff5aaeeac68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:56.133006 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcb848-8bf5-4016-aef2-3ff5aaeeac68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:56.134369 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fcb848-8bf5-4016-aef2-3ff5aaeeac68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:56.137585 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fcb848-8bf5-4016-aef2-3ff5aaeeac68 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:56.158371 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-3f57ebe3-723c-4d67-b1ec-d6f66b2a4cb5 None None] MFA Rules not processed for user `99d00c09d8374f59ac1260ade95be9ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:56.163664 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3f57ebe3-723c-4d67-b1ec-d6f66b2a4cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:56.190139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 845/1675] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:38:56.225354 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 831/1676] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:38:55 2022] POST /identity/v3/projects => generated 369 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:56.226501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57ebe3-723c-4d67-b1ec-d6f66b2a4cb5 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:56.245966 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57ebe3-723c-4d67-b1ec-d6f66b2a4cb5 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:56.271992 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcb848-8bf5-4016-aef2-3ff5aaeeac68 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:56.276861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c8fcb848-8bf5-4016-aef2-3ff5aaeeac68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:56.351223 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:56.492132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a29900-8e03-403a-8ec2-c191dc2a6219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:56.493758 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a29900-8e03-403a-8ec2-c191dc2a6219 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:56.495912 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a29900-8e03-403a-8ec2-c191dc2a6219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:56.496316 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a29900-8e03-403a-8ec2-c191dc2a6219 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:56.497103 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a29900-8e03-403a-8ec2-c191dc2a6219 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:56.501899 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a29900-8e03-403a-8ec2-c191dc2a6219 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:56.581797 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 846/1677] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:38:56 2022] POST /identity/v3/users => generated 291 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:56.587997 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a29900-8e03-403a-8ec2-c191dc2a6219 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:56.603540 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcb848-8bf5-4016-aef2-3ff5aaeeac68 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:56.628776 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a29900-8e03-403a-8ec2-c191dc2a6219 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:56.630896 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8543f15-37c0-4f2c-afda-afe6c6263fbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:56.632172 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8543f15-37c0-4f2c-afda-afe6c6263fbc service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:56.633764 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8543f15-37c0-4f2c-afda-afe6c6263fbc service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:56.634265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8543f15-37c0-4f2c-afda-afe6c6263fbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:56.656420 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5e661-8f8d-4ff0-8eae-f463390b071e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:56.657073 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5e661-8f8d-4ff0-8eae-f463390b071e admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:56.657357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5e661-8f8d-4ff0-8eae-f463390b071e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:56.657658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5e661-8f8d-4ff0-8eae-f463390b071e admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:56.664970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce5e661-8f8d-4ff0-8eae-f463390b071e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:56.677428 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce5e661-8f8d-4ff0-8eae-f463390b071e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:56.716835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 847/1678] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:56.719746 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:56.725332 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5e661-8f8d-4ff0-8eae-f463390b071e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:56.767148 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5e661-8f8d-4ff0-8eae-f463390b071e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:56.786335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92987bf-2170-483a-868d-9f5fe271e150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:56.787038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92987bf-2170-483a-868d-9f5fe271e150 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:56.787316 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92987bf-2170-483a-868d-9f5fe271e150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:56.791609 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92987bf-2170-483a-868d-9f5fe271e150 admin admin] PATH_INFO: `/v3/projects/6a040ef0ddcc487c921ca6b98d7d65aa/users/16acf1556720406ebaa30361f8e879dc/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:56.855593 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92987bf-2170-483a-868d-9f5fe271e150 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a040ef0ddcc487c921ca6b98d7d65aa, user_id=16acf1556720406ebaa30361f8e879dc, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:56.868028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92987bf-2170-483a-868d-9f5fe271e150 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:56.878272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8543f15-37c0-4f2c-afda-afe6c6263fbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:56.883162 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8543f15-37c0-4f2c-afda-afe6c6263fbc service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:56.927028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 848/1679] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:38:56 2022] PUT /identity/v3/projects/6a040ef0ddcc487c921ca6b98d7d65aa/users/16acf1556720406ebaa30361f8e879dc/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:38:56.928208 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92987bf-2170-483a-868d-9f5fe271e150 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:56.929842 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 832/1680] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:38:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:38:57.081002 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557a10c-2d1c-4711-94ea-038772dcbaf6 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:57.081301 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557a10c-2d1c-4711-94ea-038772dcbaf6 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:57.081641 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557a10c-2d1c-4711-94ea-038772dcbaf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:57.162590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92987bf-2170-483a-868d-9f5fe271e150 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:57.179689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-8557a10c-2d1c-4711-94ea-038772dcbaf6 None None] MFA Rules not processed for user `b3acc8bc894e4541bf529a7cd104c958`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:57.180883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-8557a10c-2d1c-4711-94ea-038772dcbaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:57.196745 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cdbde-ed42-4d1f-a5b2-f8e7ca63ff28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:57.197637 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cdbde-ed42-4d1f-a5b2-f8e7ca63ff28 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:57.197931 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cdbde-ed42-4d1f-a5b2-f8e7ca63ff28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:57.198207 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cdbde-ed42-4d1f-a5b2-f8e7ca63ff28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:57.198956 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1cdbde-ed42-4d1f-a5b2-f8e7ca63ff28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:57.200771 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1cdbde-ed42-4d1f-a5b2-f8e7ca63ff28 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:57.245813 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 849/1681] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:57.258182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cdbde-ed42-4d1f-a5b2-f8e7ca63ff28 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:57.276495 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cdbde-ed42-4d1f-a5b2-f8e7ca63ff28 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:57.290719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4f6d6-0016-46f5-a3ca-a70386b27a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:57.291614 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4f6d6-0016-46f5-a3ca-a70386b27a57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:57.291899 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4f6d6-0016-46f5-a3ca-a70386b27a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:57.292216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4f6d6-0016-46f5-a3ca-a70386b27a57 admin admin] PATH_INFO: `/v3/projects/6a040ef0ddcc487c921ca6b98d7d65aa/users/16acf1556720406ebaa30361f8e879dc/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:57.295542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a4f6d6-0016-46f5-a3ca-a70386b27a57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a040ef0ddcc487c921ca6b98d7d65aa, user_id=16acf1556720406ebaa30361f8e879dc, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:57.299249 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a4f6d6-0016-46f5-a3ca-a70386b27a57 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:57.316431 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 850/1682] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:38:57 2022] PUT /identity/v3/projects/6a040ef0ddcc487c921ca6b98d7d65aa/users/16acf1556720406ebaa30361f8e879dc/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:38:57.333198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15412b-08a1-412d-85ae-dfd1e50e7b3d None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:57.333701 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15412b-08a1-412d-85ae-dfd1e50e7b3d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:57.334575 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15412b-08a1-412d-85ae-dfd1e50e7b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:57.355894 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 833/1683] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:57 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:57.361264 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557a10c-2d1c-4711-94ea-038772dcbaf6 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:57.363074 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-8557a10c-2d1c-4711-94ea-038772dcbaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:57.385132 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557a10c-2d1c-4711-94ea-038772dcbaf6 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:57.422548 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7f15412b-08a1-412d-85ae-dfd1e50e7b3d None None] MFA Rules not processed for user `16acf1556720406ebaa30361f8e879dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:57.424681 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7f15412b-08a1-412d-85ae-dfd1e50e7b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:57.534882 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81684f-055e-42e3-975e-756cfbd89c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:57.536014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81684f-055e-42e3-975e-756cfbd89c4a admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:57.536311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81684f-055e-42e3-975e-756cfbd89c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:57.536615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81684f-055e-42e3-975e-756cfbd89c4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:57.537421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f81684f-055e-42e3-975e-756cfbd89c4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:57.540488 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f81684f-055e-42e3-975e-756cfbd89c4a admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:57.561228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:57.597152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 834/1684] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:38:57 2022] POST /identity/v3/projects => generated 405 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 04:38:57.602832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81684f-055e-42e3-975e-756cfbd89c4a admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:57.624688 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 851/1685] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:57 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:57.627737 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f15412b-08a1-412d-85ae-dfd1e50e7b3d None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:57.627737 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7f15412b-08a1-412d-85ae-dfd1e50e7b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:57.708723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:57.734997 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f15412b-08a1-412d-85ae-dfd1e50e7b3d None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:57.789220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81684f-055e-42e3-975e-756cfbd89c4a admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:57.796775 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7eca62-71a2-4a21-88cb-6e89617f8431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:57.797482 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7eca62-71a2-4a21-88cb-6e89617f8431 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:57.797858 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7eca62-71a2-4a21-88cb-6e89617f8431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:57.798197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7eca62-71a2-4a21-88cb-6e89617f8431 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:57.799111 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7eca62-71a2-4a21-88cb-6e89617f8431 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:57.801664 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7eca62-71a2-4a21-88cb-6e89617f8431 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:57.825758 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce206a76-2c4c-42d6-95e7-918f36ae435f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:57.829396 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce206a76-2c4c-42d6-95e7-918f36ae435f service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:57.830002 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce206a76-2c4c-42d6-95e7-918f36ae435f service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:57.830402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce206a76-2c4c-42d6-95e7-918f36ae435f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:57.838606 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 835/1686] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:38:57 2022] POST /identity/v3/users => generated 309 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:38:57.841341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9fb8b8-9435-4e7b-9856-7eca5e24a1c4 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:57.841937 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9fb8b8-9435-4e7b-9856-7eca5e24a1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:57.845639 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9fb8b8-9435-4e7b-9856-7eca5e24a1c4 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:57.847906 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 836/1687] 10.222.0.47 () {54 vars in 912 bytes} [Thu Aug 4 04:38:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:38:57.850861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9fb8b8-9435-4e7b-9856-7eca5e24a1c4 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:57.853051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-0f9fb8b8-9435-4e7b-9856-7eca5e24a1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:57.874649 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9fb8b8-9435-4e7b-9856-7eca5e24a1c4 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:57.884211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c80f0-866b-4164-aa72-d5e85aed59f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:57.886192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c80f0-866b-4164-aa72-d5e85aed59f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:57.886822 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c80f0-866b-4164-aa72-d5e85aed59f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:57.887993 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c80f0-866b-4164-aa72-d5e85aed59f0 admin admin] PATH_INFO: `/v3/users/fa8866699789473ca655509b6c3e6fe6` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:57.890722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c80f0-866b-4164-aa72-d5e85aed59f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa8866699789473ca655509b6c3e6fe6)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:57.893952 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c80f0-866b-4164-aa72-d5e85aed59f0 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:57.964121 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce206a76-2c4c-42d6-95e7-918f36ae435f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:57.966749 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce206a76-2c4c-42d6-95e7-918f36ae435f service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:57.981351 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 852/1688] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:38:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:38:57.982394 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce206a76-2c4c-42d6-95e7-918f36ae435f service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:57.995404 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce206a76-2c4c-42d6-95e7-918f36ae435f service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:58.002989 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45688ef9-c548-479b-a36e-0205021e01aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:58.003899 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45688ef9-c548-479b-a36e-0205021e01aa admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:58.004399 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45688ef9-c548-479b-a36e-0205021e01aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:58.004776 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45688ef9-c548-479b-a36e-0205021e01aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:58.005924 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45688ef9-c548-479b-a36e-0205021e01aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:58.008492 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45688ef9-c548-479b-a36e-0205021e01aa admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:58.019303 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-db3c80f0-866b-4164-aa72-d5e85aed59f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa8866699789473ca655509b6c3e6fe6'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:58.022415 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-db3c80f0-866b-4164-aa72-d5e85aed59f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa8866699789473ca655509b6c3e6fe6'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:58.036611 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 853/1689] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:58.040582 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea69ec-3324-4f6d-95f0-5fbd39e7ceee None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:58.041165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea69ec-3324-4f6d-95f0-5fbd39e7ceee None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:58.041715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea69ec-3324-4f6d-95f0-5fbd39e7ceee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:58.059916 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-db3c80f0-866b-4164-aa72-d5e85aed59f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa8866699789473ca655509b6c3e6fe6'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:58.082157 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-db3c80f0-866b-4164-aa72-d5e85aed59f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa8866699789473ca655509b6c3e6fe6'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:58.082836 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-db3c80f0-866b-4164-aa72-d5e85aed59f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa8866699789473ca655509b6c3e6fe6'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:38:58.108045 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 837/1690] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:38:57 2022] DELETE /identity/v3/users/fa8866699789473ca655509b6c3e6fe6 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:38:58.110929 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c80f0-866b-4164-aa72-d5e85aed59f0 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:58.131114 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-db3c80f0-866b-4164-aa72-d5e85aed59f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:58.143162 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-fbea69ec-3324-4f6d-95f0-5fbd39e7ceee None None] MFA Rules not processed for user `1a2a647f97fc4d369f562e8b5a4bf68c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:58.149208 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fbea69ec-3324-4f6d-95f0-5fbd39e7ceee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:58.186122 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:58.245680 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 854/1691] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Aug 4 04:38:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:58.247344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48a304-0304-4780-a2e5-a0c9cff12b40 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:58.249305 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48a304-0304-4780-a2e5-a0c9cff12b40 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:58.251510 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48a304-0304-4780-a2e5-a0c9cff12b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:58.299633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-db48a304-0304-4780-a2e5-a0c9cff12b40 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:38:58.302213 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-db48a304-0304-4780-a2e5-a0c9cff12b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:58.356006 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c80f0-866b-4164-aa72-d5e85aed59f0 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:58.366027 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a57ae-a731-4c20-bda2-4058281b2ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:58.366976 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a57ae-a731-4c20-bda2-4058281b2ece admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:58.367447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a57ae-a731-4c20-bda2-4058281b2ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:58.367911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a57ae-a731-4c20-bda2-4058281b2ece admin admin] PATH_INFO: `/v3/projects/48956e00e2cd441ea7e6ce7c30dacf0e/users/51abcce38004439a9dfd54c3622e5201/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:58.394607 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a57ae-a731-4c20-bda2-4058281b2ece admin admin] RBAC: Authorizing `identity:create_grant(project_id=48956e00e2cd441ea7e6ce7c30dacf0e, user_id=51abcce38004439a9dfd54c3622e5201, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:58.397678 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a57ae-a731-4c20-bda2-4058281b2ece admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:58.436722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 838/1692] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:38:58 2022] PUT /identity/v3/projects/48956e00e2cd441ea7e6ce7c30dacf0e/users/51abcce38004439a9dfd54c3622e5201/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:38:58.439601 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a57ae-a731-4c20-bda2-4058281b2ece admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:58.445001 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-683a57ae-a731-4c20-bda2-4058281b2ece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:58.488722 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 855/1693] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:38:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:38:58.489749 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48a304-0304-4780-a2e5-a0c9cff12b40 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:58.492064 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:38:58.524350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48a304-0304-4780-a2e5-a0c9cff12b40 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:58.605704 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a57ae-a731-4c20-bda2-4058281b2ece admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:58.613190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa6faf-601b-41cc-8a34-02c42edb43b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:58.613898 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa6faf-601b-41cc-8a34-02c42edb43b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:58.614414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa6faf-601b-41cc-8a34-02c42edb43b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:58.615000 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa6faf-601b-41cc-8a34-02c42edb43b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:58.615104 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a77b0f-452d-43e1-a769-14f6950a05e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:58.615961 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a77b0f-452d-43e1-a769-14f6950a05e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:58.616242 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a77b0f-452d-43e1-a769-14f6950a05e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:58.616531 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a77b0f-452d-43e1-a769-14f6950a05e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:58.617403 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a77b0f-452d-43e1-a769-14f6950a05e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:58.619617 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a77b0f-452d-43e1-a769-14f6950a05e5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:58.631812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa6faf-601b-41cc-8a34-02c42edb43b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:58.632975 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 856/1694] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:38:58.633809 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa6faf-601b-41cc-8a34-02c42edb43b9 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:58.634291 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a77b0f-452d-43e1-a769-14f6950a05e5 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:58.646856 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a77b0f-452d-43e1-a769-14f6950a05e5 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:58.656207 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5745b-cebf-47c4-9467-946f25461399 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:58.657097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5745b-cebf-47c4-9467-946f25461399 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:58.657380 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5745b-cebf-47c4-9467-946f25461399 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:58.657717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5745b-cebf-47c4-9467-946f25461399 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:58.678284 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 839/1695] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:38:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:38:58.679630 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5745b-cebf-47c4-9467-946f25461399 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:58.680930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa6faf-601b-41cc-8a34-02c42edb43b9 service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:58.681201 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5745b-cebf-47c4-9467-946f25461399 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:58.710346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 857/1696] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:38:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:38:58.731657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa6faf-601b-41cc-8a34-02c42edb43b9 service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:58.749342 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea61b79-7ee5-4517-b6d1-f0c34bb8b848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:38:58.750379 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea61b79-7ee5-4517-b6d1-f0c34bb8b848 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:58.750717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea61b79-7ee5-4517-b6d1-f0c34bb8b848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:58.751011 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea61b79-7ee5-4517-b6d1-f0c34bb8b848 admin admin] PATH_INFO: `/v3/projects/48956e00e2cd441ea7e6ce7c30dacf0e/users/51abcce38004439a9dfd54c3622e5201/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:58.761652 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea61b79-7ee5-4517-b6d1-f0c34bb8b848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48956e00e2cd441ea7e6ce7c30dacf0e, user_id=51abcce38004439a9dfd54c3622e5201, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:38:58.765057 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea61b79-7ee5-4517-b6d1-f0c34bb8b848 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:38:58.798496 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 840/1697] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:38:58 2022] PUT /identity/v3/projects/48956e00e2cd441ea7e6ce7c30dacf0e/users/51abcce38004439a9dfd54c3622e5201/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:38:59.805201 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5745b-cebf-47c4-9467-946f25461399 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:38:59.807961 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3ef5745b-cebf-47c4-9467-946f25461399 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:38:59.850766 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5745b-cebf-47c4-9467-946f25461399 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:38:59.958465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be78449-6152-4bd3-a719-b9226a2ade13 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:59.958917 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be78449-6152-4bd3-a719-b9226a2ade13 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:59.959080 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be78449-6152-4bd3-a719-b9226a2ade13 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:38:59.961619 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 841/1698] 10.222.0.47 () {54 vars in 961 bytes} [Thu Aug 4 04:38:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:38:59.971805 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b3082-5afc-4d93-9983-433531cfdaf4 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:38:59.972090 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b3082-5afc-4d93-9983-433531cfdaf4 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:38:59.972362 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b3082-5afc-4d93-9983-433531cfdaf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:00.032950 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-528b3082-5afc-4d93-9983-433531cfdaf4 None None] MFA Rules not processed for user `350f72f740f749f68175c1c4e8ae9d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:00.049479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-528b3082-5afc-4d93-9983-433531cfdaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:00.117270 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773b6ed-754f-43f3-9f0f-318eb2bf9779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:00.117987 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773b6ed-754f-43f3-9f0f-318eb2bf9779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:00.118261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773b6ed-754f-43f3-9f0f-318eb2bf9779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:00.118530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773b6ed-754f-43f3-9f0f-318eb2bf9779 admin admin] PATH_INFO: `/v3/projects/3c72f723819849228c8763c36ded1d11` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:00.126755 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7773b6ed-754f-43f3-9f0f-318eb2bf9779 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c72f723819849228c8763c36ded1d11)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:00.128844 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7773b6ed-754f-43f3-9f0f-318eb2bf9779 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:00.144320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:00.309579 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 842/1699] 10.222.0.47 () {58 vars in 1058 bytes} [Thu Aug 4 04:38:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:00.310384 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-7773b6ed-754f-43f3-9f0f-318eb2bf9779 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c72f723819849228c8763c36ded1d11', 'request_id': 'req-7773b6ed-754f-43f3-9f0f-318eb2bf9779'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:00.313058 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b3082-5afc-4d93-9983-433531cfdaf4 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:00.324725 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-7773b6ed-754f-43f3-9f0f-318eb2bf9779 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c72f723819849228c8763c36ded1d11', 'request_id': 'req-7773b6ed-754f-43f3-9f0f-318eb2bf9779'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:00.335969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b3082-5afc-4d93-9983-433531cfdaf4 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:00.340670 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-7773b6ed-754f-43f3-9f0f-318eb2bf9779 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c72f723819849228c8763c36ded1d11', 'request_id': 'req-7773b6ed-754f-43f3-9f0f-318eb2bf9779'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:00.377357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-7773b6ed-754f-43f3-9f0f-318eb2bf9779 admin admin] The token cache is being invalidate because project 3c72f723819849228c8763c36ded1d11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:00.377776 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-7773b6ed-754f-43f3-9f0f-318eb2bf9779 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:00.378616 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-7773b6ed-754f-43f3-9f0f-318eb2bf9779 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:00.382126 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 858/1700] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:38:59 2022] DELETE /identity/v3/projects/3c72f723819849228c8763c36ded1d11 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:00.387537 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51064ef-362d-4779-b60e-38e1b9be0d88 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:00.388055 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51064ef-362d-4779-b60e-38e1b9be0d88 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:00.392330 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51064ef-362d-4779-b60e-38e1b9be0d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:00.464771 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b51064ef-362d-4779-b60e-38e1b9be0d88 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:00.464771 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b51064ef-362d-4779-b60e-38e1b9be0d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:00.538994 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620179a2-030f-4319-82cc-167673ffb753 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350f72f740f749f68175c1c4e8ae9d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:00.539850 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620179a2-030f-4319-82cc-167673ffb753 service placement] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:00.540204 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620179a2-030f-4319-82cc-167673ffb753 service placement] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:00.541478 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620179a2-030f-4319-82cc-167673ffb753 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:00.547152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-620179a2-030f-4319-82cc-167673ffb753 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:00.606942 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620179a2-030f-4319-82cc-167673ffb753 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:00.614292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620179a2-030f-4319-82cc-167673ffb753 service placement] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:00.629793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 859/1701] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:00.642763 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c9b7b-659d-4ba3-8f06-2443aea67d0f None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:00.643347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c9b7b-659d-4ba3-8f06-2443aea67d0f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:00.649570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c9b7b-659d-4ba3-8f06-2443aea67d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:00.652350 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 843/1702] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:00.742994 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-741c9b7b-659d-4ba3-8f06-2443aea67d0f None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:00.744412 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-741c9b7b-659d-4ba3-8f06-2443aea67d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:00.781146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 860/1703] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:00.782265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620179a2-030f-4319-82cc-167673ffb753 service placement] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:00.821844 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620179a2-030f-4319-82cc-167673ffb753 service placement] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:00.851651 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886a1c9-50d8-4714-a728-8b66341be97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:00.852785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886a1c9-50d8-4714-a728-8b66341be97b admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:00.855649 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886a1c9-50d8-4714-a728-8b66341be97b admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:00.856069 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886a1c9-50d8-4714-a728-8b66341be97b admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:00.856940 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9886a1c9-50d8-4714-a728-8b66341be97b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:39:00.858274 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9886a1c9-50d8-4714-a728-8b66341be97b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:00.867031 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9886a1c9-50d8-4714-a728-8b66341be97b admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:00.911296 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 844/1704] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:39:00.915681 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c9b7b-659d-4ba3-8f06-2443aea67d0f None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:00.939510 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c9b7b-659d-4ba3-8f06-2443aea67d0f None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:00.954797 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1c599-6bbb-4bda-8eb1-623c3c8eb0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:00.955630 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1c599-6bbb-4bda-8eb1-623c3c8eb0f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:00.956050 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1c599-6bbb-4bda-8eb1-623c3c8eb0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:00.956519 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1c599-6bbb-4bda-8eb1-623c3c8eb0f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:00.957292 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1c599-6bbb-4bda-8eb1-623c3c8eb0f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:00.962482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1c599-6bbb-4bda-8eb1-623c3c8eb0f9 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:01.045102 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 861/1705] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:00 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 04:39:01.049624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886a1c9-50d8-4714-a728-8b66341be97b admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:01.066075 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886a1c9-50d8-4714-a728-8b66341be97b admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:01.152150 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:01.284724 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24a6b7-4836-4e95-9805-7f589d8bba9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:01.286751 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24a6b7-4836-4e95-9805-7f589d8bba9a admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:01.287743 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24a6b7-4836-4e95-9805-7f589d8bba9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:01.288622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24a6b7-4836-4e95-9805-7f589d8bba9a admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:01.289857 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d24a6b7-4836-4e95-9805-7f589d8bba9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:01.296210 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d24a6b7-4836-4e95-9805-7f589d8bba9a admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:01.402093 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 845/1706] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:01 2022] POST /identity/v3/users => generated 287 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:01.402928 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1c599-6bbb-4bda-8eb1-623c3c8eb0f9 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:01.426159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1c599-6bbb-4bda-8eb1-623c3c8eb0f9 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:01.446372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2947a-7f32-42bc-a1b4-dee8dac35578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:01.446921 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2947a-7f32-42bc-a1b4-dee8dac35578 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:01.447201 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2947a-7f32-42bc-a1b4-dee8dac35578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:01.447474 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2947a-7f32-42bc-a1b4-dee8dac35578 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:01.448185 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2947a-7f32-42bc-a1b4-dee8dac35578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:01.455944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2947a-7f32-42bc-a1b4-dee8dac35578 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:01.493053 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 862/1707] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:01.495032 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24a6b7-4836-4e95-9805-7f589d8bba9a admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:01.535405 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24a6b7-4836-4e95-9805-7f589d8bba9a admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:01.559562 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31266c-b4ce-4606-aaba-2639839a6051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:01.560167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31266c-b4ce-4606-aaba-2639839a6051 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:01.560459 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31266c-b4ce-4606-aaba-2639839a6051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:01.560735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31266c-b4ce-4606-aaba-2639839a6051 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:01.563303 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31266c-b4ce-4606-aaba-2639839a6051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:01.567333 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31266c-b4ce-4606-aaba-2639839a6051 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:01.611299 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 846/1708] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:01.615829 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2947a-7f32-42bc-a1b4-dee8dac35578 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:01.636410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2947a-7f32-42bc-a1b4-dee8dac35578 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:01.661248 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41598a-0d33-43eb-8e2e-1b5815ab178c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:01.662153 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41598a-0d33-43eb-8e2e-1b5815ab178c admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:01.662596 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41598a-0d33-43eb-8e2e-1b5815ab178c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:01.663026 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41598a-0d33-43eb-8e2e-1b5815ab178c admin admin] PATH_INFO: `/v3/projects/767d2090d469421baa485c5d19cd61dc/users/b05037107965471bbffe9be332758fc0/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:01.687545 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db41598a-0d33-43eb-8e2e-1b5815ab178c admin admin] RBAC: Authorizing `identity:create_grant(project_id=767d2090d469421baa485c5d19cd61dc, user_id=b05037107965471bbffe9be332758fc0, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:01.694097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db41598a-0d33-43eb-8e2e-1b5815ab178c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:01.711687 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 863/1709] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:01 2022] PUT /identity/v3/projects/767d2090d469421baa485c5d19cd61dc/users/b05037107965471bbffe9be332758fc0/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:01.723993 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592989a-9a8a-4cc9-a76f-d39fa67be7b6 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:01.723993 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592989a-9a8a-4cc9-a76f-d39fa67be7b6 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:01.723993 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592989a-9a8a-4cc9-a76f-d39fa67be7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:01.764065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41598a-0d33-43eb-8e2e-1b5815ab178c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:01.767802 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-db41598a-0d33-43eb-8e2e-1b5815ab178c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:01.815640 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-5592989a-9a8a-4cc9-a76f-d39fa67be7b6 None None] MFA Rules not processed for user `b05037107965471bbffe9be332758fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:01.816952 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5592989a-9a8a-4cc9-a76f-d39fa67be7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:01.953044 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 847/1710] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:01.959715 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592989a-9a8a-4cc9-a76f-d39fa67be7b6 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:01.966675 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5592989a-9a8a-4cc9-a76f-d39fa67be7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:01.982844 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:02.103233 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592989a-9a8a-4cc9-a76f-d39fa67be7b6 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:02.106916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41598a-0d33-43eb-8e2e-1b5815ab178c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:02.109074 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cdadbc-7081-4cac-a5bf-2e019ee53696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:02.110196 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cdadbc-7081-4cac-a5bf-2e019ee53696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:02.110975 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cdadbc-7081-4cac-a5bf-2e019ee53696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:02.111386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cdadbc-7081-4cac-a5bf-2e019ee53696 admin admin] PATH_INFO: `/v3/users/fc33224cc8164ccab1859c97cb9c24e7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:02.113772 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cdadbc-7081-4cac-a5bf-2e019ee53696 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc33224cc8164ccab1859c97cb9c24e7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:02.116536 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cdadbc-7081-4cac-a5bf-2e019ee53696 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:02.132810 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52539c0-e68c-447a-9b61-dea8d48f6375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:02.134934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52539c0-e68c-447a-9b61-dea8d48f6375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:02.135233 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52539c0-e68c-447a-9b61-dea8d48f6375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:02.135514 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52539c0-e68c-447a-9b61-dea8d48f6375 admin admin] PATH_INFO: `/v3/users/8b0ec9e9234f482e98cc15e617f181f7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:02.142990 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52539c0-e68c-447a-9b61-dea8d48f6375 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b0ec9e9234f482e98cc15e617f181f7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:02.153240 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52539c0-e68c-447a-9b61-dea8d48f6375 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:02.218955 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:02.287691 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c52539c0-e68c-447a-9b61-dea8d48f6375 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b0ec9e9234f482e98cc15e617f181f7'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:02.302711 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-e3cdadbc-7081-4cac-a5bf-2e019ee53696 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc33224cc8164ccab1859c97cb9c24e7'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:02.302711 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-e3cdadbc-7081-4cac-a5bf-2e019ee53696 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc33224cc8164ccab1859c97cb9c24e7'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:02.322266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c52539c0-e68c-447a-9b61-dea8d48f6375 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b0ec9e9234f482e98cc15e617f181f7'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:02.323905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c52539c0-e68c-447a-9b61-dea8d48f6375 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b0ec9e9234f482e98cc15e617f181f7'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:02.365099 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c52539c0-e68c-447a-9b61-dea8d48f6375 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b0ec9e9234f482e98cc15e617f181f7'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:02.366964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c52539c0-e68c-447a-9b61-dea8d48f6375 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b0ec9e9234f482e98cc15e617f181f7'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:02.379557 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-e3cdadbc-7081-4cac-a5bf-2e019ee53696 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc33224cc8164ccab1859c97cb9c24e7'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:02.402248 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 864/1711] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:01 2022] DELETE /identity/v3/users/8b0ec9e9234f482e98cc15e617f181f7 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:02.403680 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52539c0-e68c-447a-9b61-dea8d48f6375 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:02.407816 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c52539c0-e68c-447a-9b61-dea8d48f6375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:02.411592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-e3cdadbc-7081-4cac-a5bf-2e019ee53696 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc33224cc8164ccab1859c97cb9c24e7'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:02.412674 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-e3cdadbc-7081-4cac-a5bf-2e019ee53696 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc33224cc8164ccab1859c97cb9c24e7'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:02.461578 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 848/1712] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:01 2022] DELETE /identity/v3/users/fc33224cc8164ccab1859c97cb9c24e7 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:02.463825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9434e3a-2cfd-4825-8ed8-c4b8cd19a5b0 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:02.464129 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9434e3a-2cfd-4825-8ed8-c4b8cd19a5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:02.465576 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9434e3a-2cfd-4825-8ed8-c4b8cd19a5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:02.545939 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-b9434e3a-2cfd-4825-8ed8-c4b8cd19a5b0 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:02.552934 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-b9434e3a-2cfd-4825-8ed8-c4b8cd19a5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:02.591732 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52539c0-e68c-447a-9b61-dea8d48f6375 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:02.727871 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:02.741228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21abb8-8316-45c9-b4d8-7923ecbcaa15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:02.743159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21abb8-8316-45c9-b4d8-7923ecbcaa15 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:02.744758 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21abb8-8316-45c9-b4d8-7923ecbcaa15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:02.745049 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21abb8-8316-45c9-b4d8-7923ecbcaa15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:02.745844 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21abb8-8316-45c9-b4d8-7923ecbcaa15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:02.749871 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21abb8-8316-45c9-b4d8-7923ecbcaa15 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:02.817327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 849/1713] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:02.818677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be58ff-baa2-4ff3-8d68-f3fdbe775577 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:02.818962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be58ff-baa2-4ff3-8d68-f3fdbe775577 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:02.819237 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be58ff-baa2-4ff3-8d68-f3fdbe775577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:02.841277 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 865/1714] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:02 2022] POST /identity/v3/projects => generated 371 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:02.846417 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21abb8-8316-45c9-b4d8-7923ecbcaa15 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:02.872261 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-76be58ff-baa2-4ff3-8d68-f3fdbe775577 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:02.877773 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-76be58ff-baa2-4ff3-8d68-f3fdbe775577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:03.012469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 850/1715] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:03.013403 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be58ff-baa2-4ff3-8d68-f3fdbe775577 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:03.016664 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-76be58ff-baa2-4ff3-8d68-f3fdbe775577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:03.114800 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21abb8-8316-45c9-b4d8-7923ecbcaa15 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:03.138836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16030694-5055-4075-8cdc-f98aa881cd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:03.141863 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16030694-5055-4075-8cdc-f98aa881cd9c admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:03.142352 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16030694-5055-4075-8cdc-f98aa881cd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:03.142587 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16030694-5055-4075-8cdc-f98aa881cd9c admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:03.143507 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16030694-5055-4075-8cdc-f98aa881cd9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:03.145675 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be58ff-baa2-4ff3-8d68-f3fdbe775577 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:03.160236 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16030694-5055-4075-8cdc-f98aa881cd9c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:03.168856 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a6a84-882e-4207-8576-c54eaca0d4dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:03.169825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a6a84-882e-4207-8576-c54eaca0d4dc service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:03.171962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a6a84-882e-4207-8576-c54eaca0d4dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:03.172861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a6a84-882e-4207-8576-c54eaca0d4dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:03.217688 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:03.255588 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a6a84-882e-4207-8576-c54eaca0d4dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:03.257076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a6a84-882e-4207-8576-c54eaca0d4dc service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:03.268830 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 866/1716] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:02 2022] POST /identity/v3/users => generated 292 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:03.269721 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16030694-5055-4075-8cdc-f98aa881cd9c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:03.294177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 851/1717] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:03.299438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a6a84-882e-4207-8576-c54eaca0d4dc service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:03.301746 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16030694-5055-4075-8cdc-f98aa881cd9c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:03.316364 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e5268-923a-40a7-960c-9546b0347af3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:03.317101 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e5268-923a-40a7-960c-9546b0347af3 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:03.317768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e5268-923a-40a7-960c-9546b0347af3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:03.318055 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e5268-923a-40a7-960c-9546b0347af3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:03.333321 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a6a84-882e-4207-8576-c54eaca0d4dc service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:03.346824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e5268-923a-40a7-960c-9546b0347af3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:03.353618 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e5268-923a-40a7-960c-9546b0347af3 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:03.356794 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ca978-4d2d-412b-a24b-8aa6281b490b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:03.357455 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ca978-4d2d-412b-a24b-8aa6281b490b admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:03.357765 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ca978-4d2d-412b-a24b-8aa6281b490b admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:03.358784 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ca978-4d2d-412b-a24b-8aa6281b490b admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:03.359531 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ca978-4d2d-412b-a24b-8aa6281b490b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:03.376428 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ca978-4d2d-412b-a24b-8aa6281b490b admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:03.402706 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 867/1718] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:03.410109 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 852/1719] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:03.416527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e5268-923a-40a7-960c-9546b0347af3 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:03.441006 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e5268-923a-40a7-960c-9546b0347af3 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:03.463451 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ac6a6-e719-4974-a0ba-baf0b6cafaba None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:03.463746 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ac6a6-e719-4974-a0ba-baf0b6cafaba None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:03.464057 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ac6a6-e719-4974-a0ba-baf0b6cafaba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:03.487609 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930fc1f-5f54-40b3-a1f8-02ed56c932b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:03.488321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930fc1f-5f54-40b3-a1f8-02ed56c932b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:03.488603 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930fc1f-5f54-40b3-a1f8-02ed56c932b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:03.491203 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930fc1f-5f54-40b3-a1f8-02ed56c932b1 admin admin] PATH_INFO: `/v3/projects/0909643c71294b24b81d5cd767211e94/users/840adf1a45c241728428d1194f9a7b18/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:03.564295 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f930fc1f-5f54-40b3-a1f8-02ed56c932b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0909643c71294b24b81d5cd767211e94, user_id=840adf1a45c241728428d1194f9a7b18, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:03.570132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f930fc1f-5f54-40b3-a1f8-02ed56c932b1 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:03.576156 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-cd5ac6a6-e719-4974-a0ba-baf0b6cafaba None None] MFA Rules not processed for user `49a7ce9a27734dc0a187d1b217206785`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:03.587375 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-cd5ac6a6-e719-4974-a0ba-baf0b6cafaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:03.609329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 868/1720] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:03 2022] PUT /identity/v3/projects/0909643c71294b24b81d5cd767211e94/users/840adf1a45c241728428d1194f9a7b18/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:03.614386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930fc1f-5f54-40b3-a1f8-02ed56c932b1 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:03.786425 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 853/1721] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:03.795064 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930fc1f-5f54-40b3-a1f8-02ed56c932b1 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:03.803308 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ac6a6-e719-4974-a0ba-baf0b6cafaba None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:03.804588 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3976e-ffac-44c4-aafb-58ac20325be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:03.805924 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3976e-ffac-44c4-aafb-58ac20325be4 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:03.806570 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-cd5ac6a6-e719-4974-a0ba-baf0b6cafaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:03.806802 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3976e-ffac-44c4-aafb-58ac20325be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:03.808802 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3976e-ffac-44c4-aafb-58ac20325be4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:03.810700 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3976e-ffac-44c4-aafb-58ac20325be4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:03.815793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3976e-ffac-44c4-aafb-58ac20325be4 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:03.836431 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 869/1722] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 04:39:03.841425 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3976e-ffac-44c4-aafb-58ac20325be4 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:03.854238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3976e-ffac-44c4-aafb-58ac20325be4 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:03.863052 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bee8cd-71c8-4b8b-8c71-17bfa2c2054a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:03.864556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bee8cd-71c8-4b8b-8c71-17bfa2c2054a admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:03.865032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bee8cd-71c8-4b8b-8c71-17bfa2c2054a admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:03.865660 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bee8cd-71c8-4b8b-8c71-17bfa2c2054a admin admin] PATH_INFO: `/v3/projects/0909643c71294b24b81d5cd767211e94/users/840adf1a45c241728428d1194f9a7b18/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:03.868300 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bee8cd-71c8-4b8b-8c71-17bfa2c2054a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0909643c71294b24b81d5cd767211e94, user_id=840adf1a45c241728428d1194f9a7b18, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:03.871356 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bee8cd-71c8-4b8b-8c71-17bfa2c2054a admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:03.889587 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 870/1723] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:03 2022] PUT /identity/v3/projects/0909643c71294b24b81d5cd767211e94/users/840adf1a45c241728428d1194f9a7b18/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:03.905308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d79cb7-686d-4ce3-9e4f-a7c7f5f75b9e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:03.906110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d79cb7-686d-4ce3-9e4f-a7c7f5f75b9e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:03.907386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d79cb7-686d-4ce3-9e4f-a7c7f5f75b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:03.965236 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-76d79cb7-686d-4ce3-9e4f-a7c7f5f75b9e None None] MFA Rules not processed for user `840adf1a45c241728428d1194f9a7b18`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:03.969696 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-76d79cb7-686d-4ce3-9e4f-a7c7f5f75b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:03.992210 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:04.010422 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ac6a6-e719-4974-a0ba-baf0b6cafaba None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:04.174761 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 871/1724] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:03 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:04.177185 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c0034-7dfe-4cda-a316-36b3a55bd219 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:04.177185 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c0034-7dfe-4cda-a316-36b3a55bd219 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:04.177829 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c0034-7dfe-4cda-a316-36b3a55bd219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:04.198337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4b968-3f6a-4ae9-9a03-7b9630cd98c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:04.199088 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4b968-3f6a-4ae9-9a03-7b9630cd98c5 service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:04.199378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4b968-3f6a-4ae9-9a03-7b9630cd98c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:04.199659 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4b968-3f6a-4ae9-9a03-7b9630cd98c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:04.214355 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:04.240824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7b4c0034-7dfe-4cda-a316-36b3a55bd219 None None] MFA Rules not processed for user `51abcce38004439a9dfd54c3622e5201`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:04.242100 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7b4c0034-7dfe-4cda-a316-36b3a55bd219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:04.413685 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4b968-3f6a-4ae9-9a03-7b9630cd98c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:04.415896 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4b968-3f6a-4ae9-9a03-7b9630cd98c5 service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:04.440966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 854/1725] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:04.442746 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4b968-3f6a-4ae9-9a03-7b9630cd98c5 service nova] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:04.444411 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-aec4b968-3f6a-4ae9-9a03-7b9630cd98c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:04.445622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 872/1726] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:04 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:04.685853 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4b968-3f6a-4ae9-9a03-7b9630cd98c5 service nova] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:04.699928 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ddca7-75bd-420a-b40f-0c1e332a42bd None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:04.700195 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ddca7-75bd-420a-b40f-0c1e332a42bd None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:04.700530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ddca7-75bd-420a-b40f-0c1e332a42bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:04.735460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55229dd-c07f-471a-96a7-9e3b4cfa945a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:04.735806 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55229dd-c07f-471a-96a7-9e3b4cfa945a service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:04.747552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55229dd-c07f-471a-96a7-9e3b4cfa945a service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:04.748820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55229dd-c07f-471a-96a7-9e3b4cfa945a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:04.806836 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55229dd-c07f-471a-96a7-9e3b4cfa945a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:04.811486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55229dd-c07f-471a-96a7-9e3b4cfa945a service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:04.862297 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 855/1727] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:04.866691 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55229dd-c07f-471a-96a7-9e3b4cfa945a service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:04.870342 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c55229dd-c07f-471a-96a7-9e3b4cfa945a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:04.907274 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-f00ddca7-75bd-420a-b40f-0c1e332a42bd None None] MFA Rules not processed for user `3a721708c7e443e799349add27de8f84`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:04.909048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f00ddca7-75bd-420a-b40f-0c1e332a42bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:05.040942 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55229dd-c07f-471a-96a7-9e3b4cfa945a service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:05.056011 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea59005-0487-4f15-9045-aa70f62f4424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:05.056700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea59005-0487-4f15-9045-aa70f62f4424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:05.056969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea59005-0487-4f15-9045-aa70f62f4424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:05.057238 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea59005-0487-4f15-9045-aa70f62f4424 admin admin] PATH_INFO: `/v3/projects/c64a8e4fb04d43428de8807aa1f6b228` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:05.061512 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea59005-0487-4f15-9045-aa70f62f4424 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c64a8e4fb04d43428de8807aa1f6b228)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:05.067246 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea59005-0487-4f15-9045-aa70f62f4424 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:05.160482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:05.169836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 873/1728] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Aug 4 04:39:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:39:05.170982 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ddca7-75bd-420a-b40f-0c1e332a42bd None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:05.172416 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f00ddca7-75bd-420a-b40f-0c1e332a42bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:05.212632 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-7ea59005-0487-4f15-9045-aa70f62f4424 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c64a8e4fb04d43428de8807aa1f6b228', 'request_id': 'req-7ea59005-0487-4f15-9045-aa70f62f4424'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:05.221154 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:05.230967 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ddca7-75bd-420a-b40f-0c1e332a42bd None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:05.236549 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-7ea59005-0487-4f15-9045-aa70f62f4424 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c64a8e4fb04d43428de8807aa1f6b228', 'request_id': 'req-7ea59005-0487-4f15-9045-aa70f62f4424'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:05.267101 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-7ea59005-0487-4f15-9045-aa70f62f4424 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c64a8e4fb04d43428de8807aa1f6b228', 'request_id': 'req-7ea59005-0487-4f15-9045-aa70f62f4424'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:05.313926 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-7ea59005-0487-4f15-9045-aa70f62f4424 admin admin] The token cache is being invalidate because project c64a8e4fb04d43428de8807aa1f6b228 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:05.315577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-7ea59005-0487-4f15-9045-aa70f62f4424 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:05.317050 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-7ea59005-0487-4f15-9045-aa70f62f4424 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:05.323051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 856/1729] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:04 2022] DELETE /identity/v3/projects/c64a8e4fb04d43428de8807aa1f6b228 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:05.323993 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea59005-0487-4f15-9045-aa70f62f4424 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:05.329056 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-7ea59005-0487-4f15-9045-aa70f62f4424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:05.411755 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2f3a6-dd50-408c-9bd9-3010136d5e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:05.413921 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2f3a6-dd50-408c-9bd9-3010136d5e45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:05.415694 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2f3a6-dd50-408c-9bd9-3010136d5e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:05.416128 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2f3a6-dd50-408c-9bd9-3010136d5e45 admin admin] PATH_INFO: `/v3/projects/12042b3fe2fa46b8bc1cbc34728948cd` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:05.419338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2f3a6-dd50-408c-9bd9-3010136d5e45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12042b3fe2fa46b8bc1cbc34728948cd)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:05.424518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2f3a6-dd50-408c-9bd9-3010136d5e45 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:05.494178 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea59005-0487-4f15-9045-aa70f62f4424 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:05.531093 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e5a68-1287-47d3-bcb4-7cddbcacefe9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:05.532689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e5a68-1287-47d3-bcb4-7cddbcacefe9 service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:05.533023 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e5a68-1287-47d3-bcb4-7cddbcacefe9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:05.533309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e5a68-1287-47d3-bcb4-7cddbcacefe9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:05.538524 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-461e5a68-1287-47d3-bcb4-7cddbcacefe9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:05.570763 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-17d2f3a6-dd50-408c-9bd9-3010136d5e45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12042b3fe2fa46b8bc1cbc34728948cd', 'request_id': 'req-17d2f3a6-dd50-408c-9bd9-3010136d5e45'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:05.585114 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-17d2f3a6-dd50-408c-9bd9-3010136d5e45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12042b3fe2fa46b8bc1cbc34728948cd', 'request_id': 'req-17d2f3a6-dd50-408c-9bd9-3010136d5e45'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:05.608643 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-17d2f3a6-dd50-408c-9bd9-3010136d5e45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12042b3fe2fa46b8bc1cbc34728948cd', 'request_id': 'req-17d2f3a6-dd50-408c-9bd9-3010136d5e45'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:05.642590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-17d2f3a6-dd50-408c-9bd9-3010136d5e45 admin admin] The token cache is being invalidate because project 12042b3fe2fa46b8bc1cbc34728948cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:05.643045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-17d2f3a6-dd50-408c-9bd9-3010136d5e45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:05.644327 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-17d2f3a6-dd50-408c-9bd9-3010136d5e45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:05.652141 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 874/1730] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:05 2022] DELETE /identity/v3/projects/12042b3fe2fa46b8bc1cbc34728948cd => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:05.652141 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a81b2-d853-419c-8725-e27ad581e054 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:05.652141 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a81b2-d853-419c-8725-e27ad581e054 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:05.652141 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a81b2-d853-419c-8725-e27ad581e054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:05.713497 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-131a81b2-d853-419c-8725-e27ad581e054 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:05.714821 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-131a81b2-d853-419c-8725-e27ad581e054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:05.731884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e5a68-1287-47d3-bcb4-7cddbcacefe9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:05.734541 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e5a68-1287-47d3-bcb4-7cddbcacefe9 service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:05.787582 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 857/1731] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:05.788778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e5a68-1287-47d3-bcb4-7cddbcacefe9 service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:05.794217 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-461e5a68-1287-47d3-bcb4-7cddbcacefe9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:06.000856 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 875/1732] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:06.003733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432aff8-260c-44e4-a114-9e4fad4b1710 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:06.005714 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432aff8-260c-44e4-a114-9e4fad4b1710 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:06.006276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432aff8-260c-44e4-a114-9e4fad4b1710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:06.071706 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e5a68-1287-47d3-bcb4-7cddbcacefe9 service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:06.083649 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-a432aff8-260c-44e4-a114-9e4fad4b1710 None None] MFA Rules not processed for user `12f7b1c19d824ac7b8207bf001b79057`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:06.087878 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a432aff8-260c-44e4-a114-9e4fad4b1710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:06.126152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:06.228729 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:06.294729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a92398-c3fa-40b1-9142-b80e47934036 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1e539006166434790823b839bd745d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:06.295948 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a92398-c3fa-40b1-9142-b80e47934036 service swift] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:06.296671 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a92398-c3fa-40b1-9142-b80e47934036 service swift] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:06.297105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a92398-c3fa-40b1-9142-b80e47934036 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:06.299424 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-02a92398-c3fa-40b1-9142-b80e47934036 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:06.413797 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 876/1733] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:06.419917 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a432aff8-260c-44e4-a114-9e4fad4b1710 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:06.422585 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a432aff8-260c-44e4-a114-9e4fad4b1710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:06.463973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a432aff8-260c-44e4-a114-9e4fad4b1710 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:06.479478 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a92398-c3fa-40b1-9142-b80e47934036 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:06.481430 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a92398-c3fa-40b1-9142-b80e47934036 service swift] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:06.482239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cffffd-8bd2-43dd-947e-cbc2206d1553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:06.483716 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cffffd-8bd2-43dd-947e-cbc2206d1553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:06.483984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cffffd-8bd2-43dd-947e-cbc2206d1553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:06.484260 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cffffd-8bd2-43dd-947e-cbc2206d1553 admin admin] PATH_INFO: `/v3/users/16acf1556720406ebaa30361f8e879dc` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:06.491564 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cffffd-8bd2-43dd-947e-cbc2206d1553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16acf1556720406ebaa30361f8e879dc)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:06.495448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cffffd-8bd2-43dd-947e-cbc2206d1553 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:06.523902 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 858/1734] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Aug 4 04:39:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 736 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 04:39:06.527823 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae4949-163d-491c-b7f0-c7e9c7a27441 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:06.528233 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae4949-163d-491c-b7f0-c7e9c7a27441 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:06.528612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae4949-163d-491c-b7f0-c7e9c7a27441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:06.623311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-58cffffd-8bd2-43dd-947e-cbc2206d1553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16acf1556720406ebaa30361f8e879dc'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:06.640284 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-58cffffd-8bd2-43dd-947e-cbc2206d1553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16acf1556720406ebaa30361f8e879dc'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:06.641044 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-58cffffd-8bd2-43dd-947e-cbc2206d1553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16acf1556720406ebaa30361f8e879dc'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:06.647845 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-e2ae4949-163d-491c-b7f0-c7e9c7a27441 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:06.653749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-e2ae4949-163d-491c-b7f0-c7e9c7a27441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:06.666934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-58cffffd-8bd2-43dd-947e-cbc2206d1553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16acf1556720406ebaa30361f8e879dc'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:06.672008 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-58cffffd-8bd2-43dd-947e-cbc2206d1553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16acf1556720406ebaa30361f8e879dc'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:06.713310 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 859/1735] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:06.717635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae4949-163d-491c-b7f0-c7e9c7a27441 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:06.726250 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-e2ae4949-163d-491c-b7f0-c7e9c7a27441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:06.727121 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 877/1736] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:06 2022] DELETE /identity/v3/users/16acf1556720406ebaa30361f8e879dc => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:06.733664 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cffffd-8bd2-43dd-947e-cbc2206d1553 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:07.020676 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cffffd-8bd2-43dd-947e-cbc2206d1553 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:07.043274 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719293be-b69b-41cd-8d7d-fdd933ae3777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:07.043431 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719293be-b69b-41cd-8d7d-fdd933ae3777 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:07.043619 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719293be-b69b-41cd-8d7d-fdd933ae3777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:07.047113 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719293be-b69b-41cd-8d7d-fdd933ae3777 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:07.050101 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719293be-b69b-41cd-8d7d-fdd933ae3777 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:39:07.053728 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719293be-b69b-41cd-8d7d-fdd933ae3777 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:07.064601 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719293be-b69b-41cd-8d7d-fdd933ae3777 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:07.064946 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae4949-163d-491c-b7f0-c7e9c7a27441 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:07.120245 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 878/1737] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:39:07.126023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719293be-b69b-41cd-8d7d-fdd933ae3777 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:07.225958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719293be-b69b-41cd-8d7d-fdd933ae3777 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:07.269015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abacda0-e722-416d-954a-fbede4018c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:07.274446 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abacda0-e722-416d-954a-fbede4018c9d admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:07.274733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abacda0-e722-416d-954a-fbede4018c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:07.275035 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abacda0-e722-416d-954a-fbede4018c9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:07.282244 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abacda0-e722-416d-954a-fbede4018c9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:07.284393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abacda0-e722-416d-954a-fbede4018c9d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:07.398726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:07.416261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 879/1738] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:07 2022] POST /identity/v3/projects => generated 381 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:07.421237 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abacda0-e722-416d-954a-fbede4018c9d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:07.442149 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:07.494506 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8d988-c0e0-4165-a56c-6da6a0dc6bfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:07.495397 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8d988-c0e0-4165-a56c-6da6a0dc6bfb service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:07.495743 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8d988-c0e0-4165-a56c-6da6a0dc6bfb service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:07.496019 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8d988-c0e0-4165-a56c-6da6a0dc6bfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:07.504636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-91f8d988-c0e0-4165-a56c-6da6a0dc6bfb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:07.695895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abacda0-e722-416d-954a-fbede4018c9d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:07.732190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c19c1-df97-4f97-bc63-f1d20708a5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:07.733019 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c19c1-df97-4f97-bc63-f1d20708a5a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:07.733399 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c19c1-df97-4f97-bc63-f1d20708a5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:07.744900 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c19c1-df97-4f97-bc63-f1d20708a5a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:07.748078 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c19c1-df97-4f97-bc63-f1d20708a5a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:07.758775 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c19c1-df97-4f97-bc63-f1d20708a5a7 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:07.803705 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8d988-c0e0-4165-a56c-6da6a0dc6bfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:07.810097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8d988-c0e0-4165-a56c-6da6a0dc6bfb service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:07.872274 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 860/1739] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 1158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:07.877442 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 880/1740] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:07 2022] POST /identity/v3/users => generated 291 bytes in 457 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:07.880762 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8d988-c0e0-4165-a56c-6da6a0dc6bfb service nova] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:07.960573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8d988-c0e0-4165-a56c-6da6a0dc6bfb service nova] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:07.979385 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775994b2-d446-4d5d-a6b3-eaac01f06c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:07.981845 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775994b2-d446-4d5d-a6b3-eaac01f06c3e admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:07.983859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775994b2-d446-4d5d-a6b3-eaac01f06c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:07.984140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775994b2-d446-4d5d-a6b3-eaac01f06c3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:07.985053 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775994b2-d446-4d5d-a6b3-eaac01f06c3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:07.986714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775994b2-d446-4d5d-a6b3-eaac01f06c3e admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:08.030766 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 861/1741] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:07 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:08.035584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c19c1-df97-4f97-bc63-f1d20708a5a7 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:08.052861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c19c1-df97-4f97-bc63-f1d20708a5a7 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:08.079803 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611bb5c5-fcc8-4279-9863-067e737fc8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:08.080520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611bb5c5-fcc8-4279-9863-067e737fc8b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:08.080896 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611bb5c5-fcc8-4279-9863-067e737fc8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:08.081457 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611bb5c5-fcc8-4279-9863-067e737fc8b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:08.082080 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611bb5c5-fcc8-4279-9863-067e737fc8b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:08.087153 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611bb5c5-fcc8-4279-9863-067e737fc8b1 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:08.124137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 881/1742] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:08 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:08.126800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775994b2-d446-4d5d-a6b3-eaac01f06c3e admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:08.145181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775994b2-d446-4d5d-a6b3-eaac01f06c3e admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:08.165515 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d7dd4-53d0-4bb0-8f29-c190a1ec536b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:08.167689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d7dd4-53d0-4bb0-8f29-c190a1ec536b admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:08.167978 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d7dd4-53d0-4bb0-8f29-c190a1ec536b admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:08.169152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d7dd4-53d0-4bb0-8f29-c190a1ec536b admin admin] PATH_INFO: `/v3/projects/695505cc14124b4bb4378ba771624352/users/d8c974564c6f4c168cca5cec77426e98/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:08.274257 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d7dd4-53d0-4bb0-8f29-c190a1ec536b admin admin] RBAC: Authorizing `identity:create_grant(project_id=695505cc14124b4bb4378ba771624352, user_id=d8c974564c6f4c168cca5cec77426e98, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:08.277101 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d7dd4-53d0-4bb0-8f29-c190a1ec536b admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:08.317055 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 862/1743] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:08 2022] PUT /identity/v3/projects/695505cc14124b4bb4378ba771624352/users/d8c974564c6f4c168cca5cec77426e98/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:08.331424 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f0c88-ac3b-4d94-bd9d-151d64da005d None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:08.331746 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f0c88-ac3b-4d94-bd9d-151d64da005d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:08.331986 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f0c88-ac3b-4d94-bd9d-151d64da005d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:08.470173 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7b4f0c88-ac3b-4d94-bd9d-151d64da005d None None] MFA Rules not processed for user `d8c974564c6f4c168cca5cec77426e98`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:08.471516 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7b4f0c88-ac3b-4d94-bd9d-151d64da005d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:08.724488 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 882/1744] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:08.729228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d7dd4-53d0-4bb0-8f29-c190a1ec536b admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:08.971990 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d7dd4-53d0-4bb0-8f29-c190a1ec536b admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:08.987172 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe13973-c180-40e2-ad66-3388cecf3e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:08.988382 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe13973-c180-40e2-ad66-3388cecf3e2e admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:08.988636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe13973-c180-40e2-ad66-3388cecf3e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:08.989036 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe13973-c180-40e2-ad66-3388cecf3e2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:08.995166 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe13973-c180-40e2-ad66-3388cecf3e2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:09.003694 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe13973-c180-40e2-ad66-3388cecf3e2e admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:09.106485 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 863/1745] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:08 2022] POST /identity/v3/projects => generated 381 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:09.106720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f0c88-ac3b-4d94-bd9d-151d64da005d None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:09.126547 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f0c88-ac3b-4d94-bd9d-151d64da005d None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:09.466827 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d1467-2038-4a2f-9f7d-e383898afacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:09.474296 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d1467-2038-4a2f-9f7d-e383898afacc admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:09.474296 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d1467-2038-4a2f-9f7d-e383898afacc admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:09.474296 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d1467-2038-4a2f-9f7d-e383898afacc admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:09.474296 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99d1467-2038-4a2f-9f7d-e383898afacc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:09.474296 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99d1467-2038-4a2f-9f7d-e383898afacc admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:09.545250 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe13973-c180-40e2-ad66-3388cecf3e2e admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:09.556051 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 883/1746] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:09 2022] POST /identity/v3/users => generated 297 bytes in 454 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 04:39:09.559135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe13973-c180-40e2-ad66-3388cecf3e2e admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:09.569953 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d115e-4ac2-47a8-b88d-5cc31d0c3552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:09.571415 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d115e-4ac2-47a8-b88d-5cc31d0c3552 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:09.572049 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d115e-4ac2-47a8-b88d-5cc31d0c3552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:09.572516 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d115e-4ac2-47a8-b88d-5cc31d0c3552 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:09.573522 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d115e-4ac2-47a8-b88d-5cc31d0c3552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:09.575748 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d115e-4ac2-47a8-b88d-5cc31d0c3552 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:09.602492 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d1467-2038-4a2f-9f7d-e383898afacc admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:09.611070 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 864/1747] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:09.611288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:09.620442 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d1467-2038-4a2f-9f7d-e383898afacc admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:09.645029 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40315f93-6cd7-47ad-85ab-c697435db7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:09.647505 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40315f93-6cd7-47ad-85ab-c697435db7a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:09.648930 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40315f93-6cd7-47ad-85ab-c697435db7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:09.650114 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40315f93-6cd7-47ad-85ab-c697435db7a8 admin admin] PATH_INFO: `/v3/projects/6283c7287061499bbbd70830190ce820/users/59c0b3df618d40948b8ab1f033cf5089/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:09.711771 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40315f93-6cd7-47ad-85ab-c697435db7a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6283c7287061499bbbd70830190ce820, user_id=59c0b3df618d40948b8ab1f033cf5089, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:09.719667 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40315f93-6cd7-47ad-85ab-c697435db7a8 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:09.776858 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d115e-4ac2-47a8-b88d-5cc31d0c3552 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:09.777437 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 884/1748] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:09 2022] PUT /identity/v3/projects/6283c7287061499bbbd70830190ce820/users/59c0b3df618d40948b8ab1f033cf5089/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:09.800001 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d115e-4ac2-47a8-b88d-5cc31d0c3552 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:09.906933 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:10.013845 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b294af4-0814-4966-bf4a-02b75abba62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:10.014835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b294af4-0814-4966-bf4a-02b75abba62c admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:10.015187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b294af4-0814-4966-bf4a-02b75abba62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:10.015616 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b294af4-0814-4966-bf4a-02b75abba62c admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:10.037330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b294af4-0814-4966-bf4a-02b75abba62c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:10.048491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b294af4-0814-4966-bf4a-02b75abba62c admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:10.122018 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 865/1749] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:10.127005 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40315f93-6cd7-47ad-85ab-c697435db7a8 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:10.174220 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40315f93-6cd7-47ad-85ab-c697435db7a8 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:10.196525 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926db99-fa0e-4c7e-894c-dacc33cacab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:10.199287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926db99-fa0e-4c7e-894c-dacc33cacab9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:10.201988 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926db99-fa0e-4c7e-894c-dacc33cacab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:10.202313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926db99-fa0e-4c7e-894c-dacc33cacab9 admin admin] PATH_INFO: `/v3/projects/6283c7287061499bbbd70830190ce820/users/59c0b3df618d40948b8ab1f033cf5089/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:10.211947 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926db99-fa0e-4c7e-894c-dacc33cacab9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6283c7287061499bbbd70830190ce820, user_id=59c0b3df618d40948b8ab1f033cf5089, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:10.283077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38666f0-0f1f-4572-8ec7-27ca129813ad None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:10.283517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38666f0-0f1f-4572-8ec7-27ca129813ad None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:10.284832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38666f0-0f1f-4572-8ec7-27ca129813ad None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:10.286430 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 866/1750] 10.222.0.47 () {54 vars in 921 bytes} [Thu Aug 4 04:39:10 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:39:10.289523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926db99-fa0e-4c7e-894c-dacc33cacab9 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:10.321297 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38666f0-0f1f-4572-8ec7-27ca129813ad None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:10.325300 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-a38666f0-0f1f-4572-8ec7-27ca129813ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:10.362191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 885/1751] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:10 2022] PUT /identity/v3/projects/6283c7287061499bbbd70830190ce820/users/59c0b3df618d40948b8ab1f033cf5089/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:10.374773 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f347e-bced-48a5-b767-6cb46c421229 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:10.374983 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f347e-bced-48a5-b767-6cb46c421229 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:10.375477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f347e-bced-48a5-b767-6cb46c421229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:10.479019 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-cc9f347e-bced-48a5-b767-6cb46c421229 None None] MFA Rules not processed for user `59c0b3df618d40948b8ab1f033cf5089`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:10.480281 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-cc9f347e-bced-48a5-b767-6cb46c421229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:10.550182 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38666f0-0f1f-4572-8ec7-27ca129813ad None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:10.575611 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33925a39-2505-4966-8711-b422c0a3fdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:10.577253 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33925a39-2505-4966-8711-b422c0a3fdb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:10.577724 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33925a39-2505-4966-8711-b422c0a3fdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:10.578157 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33925a39-2505-4966-8711-b422c0a3fdb5 admin admin] PATH_INFO: `/v3/projects/6a040ef0ddcc487c921ca6b98d7d65aa` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:10.580678 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33925a39-2505-4966-8711-b422c0a3fdb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a040ef0ddcc487c921ca6b98d7d65aa)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:10.582822 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33925a39-2505-4966-8711-b422c0a3fdb5 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:10.777252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 886/1752] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:39:10.778634 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dde3a-9d25-4d8f-ae65-837f6c91104d None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:10.778926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dde3a-9d25-4d8f-ae65-837f6c91104d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:10.779205 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dde3a-9d25-4d8f-ae65-837f6c91104d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:10.827627 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ff4dde3a-9d25-4d8f-ae65-837f6c91104d None None] MFA Rules not processed for user `350f72f740f749f68175c1c4e8ae9d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:10.828901 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ff4dde3a-9d25-4d8f-ae65-837f6c91104d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:10.892648 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-33925a39-2505-4966-8711-b422c0a3fdb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a040ef0ddcc487c921ca6b98d7d65aa', 'request_id': 'req-33925a39-2505-4966-8711-b422c0a3fdb5'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:10.942620 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-33925a39-2505-4966-8711-b422c0a3fdb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a040ef0ddcc487c921ca6b98d7d65aa', 'request_id': 'req-33925a39-2505-4966-8711-b422c0a3fdb5'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:10.965086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-33925a39-2505-4966-8711-b422c0a3fdb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a040ef0ddcc487c921ca6b98d7d65aa', 'request_id': 'req-33925a39-2505-4966-8711-b422c0a3fdb5'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:11.025815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-33925a39-2505-4966-8711-b422c0a3fdb5 admin admin] The token cache is being invalidate because project 6a040ef0ddcc487c921ca6b98d7d65aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:11.028436 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-33925a39-2505-4966-8711-b422c0a3fdb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:11.031593 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-33925a39-2505-4966-8711-b422c0a3fdb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:11.044164 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 867/1753] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:10 2022] DELETE /identity/v3/projects/6a040ef0ddcc487c921ca6b98d7d65aa => generated 0 bytes in 718 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:11.050720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33925a39-2505-4966-8711-b422c0a3fdb5 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:11.055672 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-33925a39-2505-4966-8711-b422c0a3fdb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:11.060227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 887/1754] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Aug 4 04:39:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:11.064041 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8564d79-8fbe-4400-a146-bfd56f7ce455 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:11.064461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8564d79-8fbe-4400-a146-bfd56f7ce455 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:11.065794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8564d79-8fbe-4400-a146-bfd56f7ce455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:11.113643 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b8564d79-8fbe-4400-a146-bfd56f7ce455 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:11.115026 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b8564d79-8fbe-4400-a146-bfd56f7ce455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:11.202204 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:11.220614 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33925a39-2505-4966-8711-b422c0a3fdb5 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:11.226474 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c733e0-1923-46f2-a1ba-cfb7308d093a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:11.227389 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c733e0-1923-46f2-a1ba-cfb7308d093a service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:11.228120 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c733e0-1923-46f2-a1ba-cfb7308d093a service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:11.228933 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c733e0-1923-46f2-a1ba-cfb7308d093a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:11.231860 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-13c733e0-1923-46f2-a1ba-cfb7308d093a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:11.241387 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:11.255789 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 888/1755] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:11.256797 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8564d79-8fbe-4400-a146-bfd56f7ce455 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:11.258069 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b8564d79-8fbe-4400-a146-bfd56f7ce455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:11.304871 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8564d79-8fbe-4400-a146-bfd56f7ce455 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:11.319866 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea0689-1dba-4bc1-9f89-dbf1b0062fd2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350f72f740f749f68175c1c4e8ae9d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:11.320175 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea0689-1dba-4bc1-9f89-dbf1b0062fd2 service placement] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:11.320463 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea0689-1dba-4bc1-9f89-dbf1b0062fd2 service placement] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:11.320740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea0689-1dba-4bc1-9f89-dbf1b0062fd2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:11.335634 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea0689-1dba-4bc1-9f89-dbf1b0062fd2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:11.337286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea0689-1dba-4bc1-9f89-dbf1b0062fd2 service placement] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:11.361057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 889/1756] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:11.364260 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d327afd-7d43-4ed3-80b7-a6bf7163fa45 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:11.364260 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d327afd-7d43-4ed3-80b7-a6bf7163fa45 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:11.366143 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d327afd-7d43-4ed3-80b7-a6bf7163fa45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:11.408425 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-6d327afd-7d43-4ed3-80b7-a6bf7163fa45 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:11.410692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6d327afd-7d43-4ed3-80b7-a6bf7163fa45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:11.428086 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 890/1757] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:11.432247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d327afd-7d43-4ed3-80b7-a6bf7163fa45 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:11.450248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c733e0-1923-46f2-a1ba-cfb7308d093a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:11.451939 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c733e0-1923-46f2-a1ba-cfb7308d093a service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:11.469064 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d327afd-7d43-4ed3-80b7-a6bf7163fa45 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:11.469321 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 868/1758] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:11 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:11.479729 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0a09c-9268-4863-b14f-58accb977de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:11.480369 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0a09c-9268-4863-b14f-58accb977de3 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:11.480654 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0a09c-9268-4863-b14f-58accb977de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:11.480930 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0a09c-9268-4863-b14f-58accb977de3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:11.484379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0a09c-9268-4863-b14f-58accb977de3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:39:11.485572 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0a09c-9268-4863-b14f-58accb977de3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:11.487289 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0a09c-9268-4863-b14f-58accb977de3 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:11.551501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 891/1759] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:39:11.552763 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c733e0-1923-46f2-a1ba-cfb7308d093a service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:11.574306 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c733e0-1923-46f2-a1ba-cfb7308d093a service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:11.584958 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43e5db-1fa9-46d1-bee4-11cd925d7a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:11.585842 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43e5db-1fa9-46d1-bee4-11cd925d7a07 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:11.586675 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43e5db-1fa9-46d1-bee4-11cd925d7a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:11.587197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43e5db-1fa9-46d1-bee4-11cd925d7a07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:11.588139 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d43e5db-1fa9-46d1-bee4-11cd925d7a07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:11.590385 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d43e5db-1fa9-46d1-bee4-11cd925d7a07 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:11.660308 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 869/1760] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:11 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:11.664891 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0a09c-9268-4863-b14f-58accb977de3 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:11.686679 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0a09c-9268-4863-b14f-58accb977de3 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:11.739667 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43e5db-1fa9-46d1-bee4-11cd925d7a07 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:11.890478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29322e08-603f-4a60-a37c-c904161d6feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:11.891122 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29322e08-603f-4a60-a37c-c904161d6feb admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:11.891408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29322e08-603f-4a60-a37c-c904161d6feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:11.891706 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29322e08-603f-4a60-a37c-c904161d6feb admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:11.892405 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29322e08-603f-4a60-a37c-c904161d6feb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:11.894526 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29322e08-603f-4a60-a37c-c904161d6feb admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:11.910927 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43e5db-1fa9-46d1-bee4-11cd925d7a07 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:11.932088 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e5d69-d17c-4b2d-a6fb-7cac1e6d724e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:11.932753 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e5d69-d17c-4b2d-a6fb-7cac1e6d724e service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:11.933036 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e5d69-d17c-4b2d-a6fb-7cac1e6d724e service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:11.933309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e5d69-d17c-4b2d-a6fb-7cac1e6d724e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:11.940495 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-6c9e5d69-d17c-4b2d-a6fb-7cac1e6d724e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:11.947303 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 892/1761] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:11 2022] POST /identity/v3/users => generated 293 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:11.958686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29322e08-603f-4a60-a37c-c904161d6feb admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:11.978030 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29322e08-603f-4a60-a37c-c904161d6feb admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:12.000801 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54603b3-6ea2-4221-b438-9ff8ba968a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:12.001462 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54603b3-6ea2-4221-b438-9ff8ba968a67 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:12.001789 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54603b3-6ea2-4221-b438-9ff8ba968a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:12.002064 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54603b3-6ea2-4221-b438-9ff8ba968a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:12.003111 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54603b3-6ea2-4221-b438-9ff8ba968a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:12.004501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54603b3-6ea2-4221-b438-9ff8ba968a67 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:12.046111 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 893/1762] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:12.051247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54603b3-6ea2-4221-b438-9ff8ba968a67 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:12.075200 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54603b3-6ea2-4221-b438-9ff8ba968a67 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:12.080697 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c76109-1218-460c-9e89-4df01e66440c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:12.081338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c76109-1218-460c-9e89-4df01e66440c admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:12.081736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c76109-1218-460c-9e89-4df01e66440c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:12.082018 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c76109-1218-460c-9e89-4df01e66440c admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:12.082798 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c76109-1218-460c-9e89-4df01e66440c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:12.084409 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c76109-1218-460c-9e89-4df01e66440c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:12.098502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 894/1763] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:12.100196 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c76109-1218-460c-9e89-4df01e66440c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:12.116455 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c76109-1218-460c-9e89-4df01e66440c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:12.126100 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc08caf-1c81-4d6b-ac48-52301d531596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:12.127251 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc08caf-1c81-4d6b-ac48-52301d531596 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:12.127763 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc08caf-1c81-4d6b-ac48-52301d531596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:12.128273 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc08caf-1c81-4d6b-ac48-52301d531596 admin admin] PATH_INFO: `/v3/projects/56d36a2533754402857d34d87b48e032/users/b170acb342a24b58a044c78231fe52ac/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:12.130284 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9e5d69-d17c-4b2d-a6fb-7cac1e6d724e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:12.131682 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9e5d69-d17c-4b2d-a6fb-7cac1e6d724e service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:12.151691 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 870/1764] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:39:12.168799 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc08caf-1c81-4d6b-ac48-52301d531596 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56d36a2533754402857d34d87b48e032, user_id=b170acb342a24b58a044c78231fe52ac, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:12.173879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc08caf-1c81-4d6b-ac48-52301d531596 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:12.205728 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 895/1765] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:12 2022] PUT /identity/v3/projects/56d36a2533754402857d34d87b48e032/users/b170acb342a24b58a044c78231fe52ac/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:12.216314 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98a9e4-0c6e-465b-904c-7c0718ca109b None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:12.217044 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98a9e4-0c6e-465b-904c-7c0718ca109b None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:12.217733 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98a9e4-0c6e-465b-904c-7c0718ca109b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:12.277152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-7c98a9e4-0c6e-465b-904c-7c0718ca109b None None] MFA Rules not processed for user `b170acb342a24b58a044c78231fe52ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:12.279849 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-7c98a9e4-0c6e-465b-904c-7c0718ca109b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:12.334392 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:12.355265 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc08caf-1c81-4d6b-ac48-52301d531596 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:12.358409 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fdc08caf-1c81-4d6b-ac48-52301d531596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:12.497456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 871/1766] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:12.500075 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98a9e4-0c6e-465b-904c-7c0718ca109b None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:12.696347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:12.710445 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc08caf-1c81-4d6b-ac48-52301d531596 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:12.755022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925f98d-a817-4428-981a-056e45eb806e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:12.756368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925f98d-a817-4428-981a-056e45eb806e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:12.757368 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98a9e4-0c6e-465b-904c-7c0718ca109b None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:12.758079 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925f98d-a817-4428-981a-056e45eb806e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:12.758394 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925f98d-a817-4428-981a-056e45eb806e admin admin] PATH_INFO: `/v3/users/b3acc8bc894e4541bf529a7cd104c958` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:12.766779 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b925f98d-a817-4428-981a-056e45eb806e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3acc8bc894e4541bf529a7cd104c958)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:12.768812 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b925f98d-a817-4428-981a-056e45eb806e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:12.777186 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687d9b1f-2ef8-4f51-9295-596e29a9551f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:12.778207 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d9b1f-2ef8-4f51-9295-596e29a9551f admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:12.778340 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d9b1f-2ef8-4f51-9295-596e29a9551f admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:12.778608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d9b1f-2ef8-4f51-9295-596e29a9551f admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:12.779374 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687d9b1f-2ef8-4f51-9295-596e29a9551f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:12.781848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687d9b1f-2ef8-4f51-9295-596e29a9551f admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:12.894079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 872/1767] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:12 2022] POST /identity/v3/projects => generated 385 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:12.896011 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857638db-a6a6-4812-9027-be6b2e4dfb2a None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:12.896406 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857638db-a6a6-4812-9027-be6b2e4dfb2a None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:12.896785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857638db-a6a6-4812-9027-be6b2e4dfb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:12.931655 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b925f98d-a817-4428-981a-056e45eb806e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3acc8bc894e4541bf529a7cd104c958'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:12.975817 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-857638db-a6a6-4812-9027-be6b2e4dfb2a None None] MFA Rules not processed for user `49a7ce9a27734dc0a187d1b217206785`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:12.977244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-857638db-a6a6-4812-9027-be6b2e4dfb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:12.990342 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b925f98d-a817-4428-981a-056e45eb806e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3acc8bc894e4541bf529a7cd104c958'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:12.991184 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b925f98d-a817-4428-981a-056e45eb806e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3acc8bc894e4541bf529a7cd104c958'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:13.062269 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b925f98d-a817-4428-981a-056e45eb806e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3acc8bc894e4541bf529a7cd104c958'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:13.062875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b925f98d-a817-4428-981a-056e45eb806e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3acc8bc894e4541bf529a7cd104c958'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:13.110013 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 896/1768] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:12 2022] DELETE /identity/v3/users/b3acc8bc894e4541bf529a7cd104c958 => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:13.111910 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925f98d-a817-4428-981a-056e45eb806e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:13.127410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b925f98d-a817-4428-981a-056e45eb806e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:13.178524 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 873/1769] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:13.178524 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6772c-319a-4660-b003-12d856596bbb None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:13.178524 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6772c-319a-4660-b003-12d856596bbb None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:13.178524 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6772c-319a-4660-b003-12d856596bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:13.280501 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-52a6772c-319a-4660-b003-12d856596bbb None None] MFA Rules not processed for user `1a2a647f97fc4d369f562e8b5a4bf68c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:13.282077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-52a6772c-319a-4660-b003-12d856596bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:13.333827 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925f98d-a817-4428-981a-056e45eb806e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:13.339044 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6803f1-2c29-4bce-89ea-b21c82d856f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:13.339693 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6803f1-2c29-4bce-89ea-b21c82d856f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:13.339979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6803f1-2c29-4bce-89ea-b21c82d856f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:13.340364 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6803f1-2c29-4bce-89ea-b21c82d856f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:13.341101 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6803f1-2c29-4bce-89ea-b21c82d856f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:13.344825 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6803f1-2c29-4bce-89ea-b21c82d856f5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:13.403969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 897/1770] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:13 2022] POST /identity/v3/users => generated 299 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:13.415495 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6803f1-2c29-4bce-89ea-b21c82d856f5 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:13.481213 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 874/1771] 10.222.0.65 () {58 vars in 1015 bytes} [Thu Aug 4 04:39:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:13.483845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6803f1-2c29-4bce-89ea-b21c82d856f5 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:13.498502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223250e-b53a-443a-81de-47508e74c470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:13.499160 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223250e-b53a-443a-81de-47508e74c470 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:13.499452 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223250e-b53a-443a-81de-47508e74c470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:13.506688 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223250e-b53a-443a-81de-47508e74c470 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:13.508846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e223250e-b53a-443a-81de-47508e74c470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:13.521082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e223250e-b53a-443a-81de-47508e74c470 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:13.534014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6772c-319a-4660-b003-12d856596bbb None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:13.535853 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-52a6772c-319a-4660-b003-12d856596bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:13.553403 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:13.563743 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 898/1772] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:13.569205 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223250e-b53a-443a-81de-47508e74c470 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:13.570504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6772c-319a-4660-b003-12d856596bbb None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:13.587258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8532b4-a305-4379-b666-660fe3d826da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:13.588464 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8532b4-a305-4379-b666-660fe3d826da service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:13.593528 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8532b4-a305-4379-b666-660fe3d826da service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:13.594135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8532b4-a305-4379-b666-660fe3d826da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:13.594470 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223250e-b53a-443a-81de-47508e74c470 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:13.613853 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcebbd91-273c-4fbc-be48-5fcf93abe0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:13.615927 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcebbd91-273c-4fbc-be48-5fcf93abe0ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:13.617508 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcebbd91-273c-4fbc-be48-5fcf93abe0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:13.620482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcebbd91-273c-4fbc-be48-5fcf93abe0ff admin admin] PATH_INFO: `/v3/projects/926952a897c8470fb3ef11180dbea64f/users/a1b2c14ca679487bb3636b69db47e2d6/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:13.626220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8532b4-a305-4379-b666-660fe3d826da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:13.651532 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8532b4-a305-4379-b666-660fe3d826da service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:13.679398 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcebbd91-273c-4fbc-be48-5fcf93abe0ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=926952a897c8470fb3ef11180dbea64f, user_id=a1b2c14ca679487bb3636b69db47e2d6, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:13.694367 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcebbd91-273c-4fbc-be48-5fcf93abe0ff admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:13.694631 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 875/1773] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:39:13.741207 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 899/1774] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:13 2022] PUT /identity/v3/projects/926952a897c8470fb3ef11180dbea64f/users/a1b2c14ca679487bb3636b69db47e2d6/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:13.745635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8532b4-a305-4379-b666-660fe3d826da service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:13.777248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8532b4-a305-4379-b666-660fe3d826da service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:14.006654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a059b-e05c-47de-ad24-4e131d047060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:14.020165 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a059b-e05c-47de-ad24-4e131d047060 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:14.020165 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a059b-e05c-47de-ad24-4e131d047060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:14.020165 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a059b-e05c-47de-ad24-4e131d047060 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:14.020165 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a059b-e05c-47de-ad24-4e131d047060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:14.024470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a059b-e05c-47de-ad24-4e131d047060 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:14.092790 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcebbd91-273c-4fbc-be48-5fcf93abe0ff admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:14.106186 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 876/1775] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:14.109638 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8dd897-dbe3-4cad-8b4b-403c90c8baea None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:14.111338 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8dd897-dbe3-4cad-8b4b-403c90c8baea None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:14.112561 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcebbd91-273c-4fbc-be48-5fcf93abe0ff admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:14.112821 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8dd897-dbe3-4cad-8b4b-403c90c8baea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:14.152294 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ba0c4-1a65-46dd-aaeb-344f7da31cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:14.152294 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ba0c4-1a65-46dd-aaeb-344f7da31cef admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:14.152294 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ba0c4-1a65-46dd-aaeb-344f7da31cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:14.152294 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ba0c4-1a65-46dd-aaeb-344f7da31cef admin admin] PATH_INFO: `/v3/projects/926952a897c8470fb3ef11180dbea64f/users/a1b2c14ca679487bb3636b69db47e2d6/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:14.159695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ba0c4-1a65-46dd-aaeb-344f7da31cef admin admin] RBAC: Authorizing `identity:create_grant(project_id=926952a897c8470fb3ef11180dbea64f, user_id=a1b2c14ca679487bb3636b69db47e2d6, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:14.171550 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ba0c4-1a65-46dd-aaeb-344f7da31cef admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:14.214019 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-3d8dd897-dbe3-4cad-8b4b-403c90c8baea None None] MFA Rules not processed for user `12f7b1c19d824ac7b8207bf001b79057`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:14.218503 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-3d8dd897-dbe3-4cad-8b4b-403c90c8baea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:14.222135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 900/1776] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:14 2022] PUT /identity/v3/projects/926952a897c8470fb3ef11180dbea64f/users/a1b2c14ca679487bb3636b69db47e2d6/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:14.236546 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc8130-044b-4f0d-8831-229c3ee3af05 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:14.236805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc8130-044b-4f0d-8831-229c3ee3af05 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:14.237084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc8130-044b-4f0d-8831-229c3ee3af05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:14.333454 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-29fc8130-044b-4f0d-8831-229c3ee3af05 None None] MFA Rules not processed for user `a1b2c14ca679487bb3636b69db47e2d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:14.339566 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-29fc8130-044b-4f0d-8831-229c3ee3af05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:14.356861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=84,keystone:DELETE=10,keystone:INSERT=6 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:14.403968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 877/1777] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:14.416354 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8dd897-dbe3-4cad-8b4b-403c90c8baea None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:14.418964 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-3d8dd897-dbe3-4cad-8b4b-403c90c8baea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:14.557391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 901/1778] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:14.578848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8dd897-dbe3-4cad-8b4b-403c90c8baea None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:14.595058 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae10cd7-be92-40be-abfa-5e3cd30332de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:14.596019 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae10cd7-be92-40be-abfa-5e3cd30332de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:14.596462 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae10cd7-be92-40be-abfa-5e3cd30332de admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:14.596893 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae10cd7-be92-40be-abfa-5e3cd30332de admin admin] PATH_INFO: `/v3/projects/7598ff62196641f39835ee1b894f151e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:14.602307 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae10cd7-be92-40be-abfa-5e3cd30332de admin admin] RBAC: Authorizing `identity:delete_project(project_id=7598ff62196641f39835ee1b894f151e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:14.604554 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae10cd7-be92-40be-abfa-5e3cd30332de admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:14.704935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:14.726850 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-0ae10cd7-be92-40be-abfa-5e3cd30332de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7598ff62196641f39835ee1b894f151e', 'request_id': 'req-0ae10cd7-be92-40be-abfa-5e3cd30332de'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:14.743209 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-0ae10cd7-be92-40be-abfa-5e3cd30332de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7598ff62196641f39835ee1b894f151e', 'request_id': 'req-0ae10cd7-be92-40be-abfa-5e3cd30332de'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:14.751970 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-0ae10cd7-be92-40be-abfa-5e3cd30332de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7598ff62196641f39835ee1b894f151e', 'request_id': 'req-0ae10cd7-be92-40be-abfa-5e3cd30332de'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:14.786151 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-0ae10cd7-be92-40be-abfa-5e3cd30332de admin admin] The token cache is being invalidate because project 7598ff62196641f39835ee1b894f151e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:14.786445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-0ae10cd7-be92-40be-abfa-5e3cd30332de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:14.800706 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-0ae10cd7-be92-40be-abfa-5e3cd30332de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:14.806498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 878/1779] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:14 2022] DELETE /identity/v3/projects/7598ff62196641f39835ee1b894f151e => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:14.812430 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc8130-044b-4f0d-8831-229c3ee3af05 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:14.840358 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc8130-044b-4f0d-8831-229c3ee3af05 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:14.928159 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae10cd7-be92-40be-abfa-5e3cd30332de admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:14.929669 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-0ae10cd7-be92-40be-abfa-5e3cd30332de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:15.007073 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8924e-9816-4681-9c7d-9f11abf123a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:15.007898 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8924e-9816-4681-9c7d-9f11abf123a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:15.008332 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8924e-9816-4681-9c7d-9f11abf123a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:15.008746 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8924e-9816-4681-9c7d-9f11abf123a6 admin admin] PATH_INFO: `/v3/users/51abcce38004439a9dfd54c3622e5201` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:15.010483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8924e-9816-4681-9c7d-9f11abf123a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51abcce38004439a9dfd54c3622e5201)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:15.012790 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8924e-9816-4681-9c7d-9f11abf123a6 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:15.106101 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-05a8924e-9816-4681-9c7d-9f11abf123a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51abcce38004439a9dfd54c3622e5201'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:15.117231 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-05a8924e-9816-4681-9c7d-9f11abf123a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51abcce38004439a9dfd54c3622e5201'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:15.118192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-05a8924e-9816-4681-9c7d-9f11abf123a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51abcce38004439a9dfd54c3622e5201'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:15.164364 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-05a8924e-9816-4681-9c7d-9f11abf123a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51abcce38004439a9dfd54c3622e5201'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:15.168066 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae10cd7-be92-40be-abfa-5e3cd30332de admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:15.168786 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-05a8924e-9816-4681-9c7d-9f11abf123a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51abcce38004439a9dfd54c3622e5201'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:15.185009 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f1357-4673-41ad-876d-302c08902a64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:15.188824 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f1357-4673-41ad-876d-302c08902a64 service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:15.190333 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f1357-4673-41ad-876d-302c08902a64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:15.190681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f1357-4673-41ad-876d-302c08902a64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:15.196270 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-b60f1357-4673-41ad-876d-302c08902a64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:15.208522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 902/1780] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:14 2022] DELETE /identity/v3/users/51abcce38004439a9dfd54c3622e5201 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:15.324032 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f1357-4673-41ad-876d-302c08902a64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:15.327570 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f1357-4673-41ad-876d-302c08902a64 service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:15.364737 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 879/1781] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:14 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:15.557662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8924e-9816-4681-9c7d-9f11abf123a6 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:15.559415 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-05a8924e-9816-4681-9c7d-9f11abf123a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:15.667136 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:15.800494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8924e-9816-4681-9c7d-9f11abf123a6 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:15.818295 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1525ca-6a9a-4093-b5d2-321dc5080191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:15.819058 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1525ca-6a9a-4093-b5d2-321dc5080191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:15.819397 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1525ca-6a9a-4093-b5d2-321dc5080191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:15.819509 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1525ca-6a9a-4093-b5d2-321dc5080191 admin admin] PATH_INFO: `/v3/users/b170acb342a24b58a044c78231fe52ac` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:15.823607 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1525ca-6a9a-4093-b5d2-321dc5080191 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b170acb342a24b58a044c78231fe52ac)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:15.826913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1525ca-6a9a-4093-b5d2-321dc5080191 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:15.882598 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f1357-4673-41ad-876d-302c08902a64 service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:15.884266 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-b60f1357-4673-41ad-876d-302c08902a64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:15.912885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f1357-4673-41ad-876d-302c08902a64 service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:15.922424 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad327a-e076-4d0e-b8a4-d2bbd7bdd619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:15.923183 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad327a-e076-4d0e-b8a4-d2bbd7bdd619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:15.923569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad327a-e076-4d0e-b8a4-d2bbd7bdd619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:15.923957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad327a-e076-4d0e-b8a4-d2bbd7bdd619 admin admin] PATH_INFO: `/v3/projects/48956e00e2cd441ea7e6ce7c30dacf0e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:15.925296 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ad327a-e076-4d0e-b8a4-d2bbd7bdd619 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48956e00e2cd441ea7e6ce7c30dacf0e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:15.928341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ad327a-e076-4d0e-b8a4-d2bbd7bdd619 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:16.005261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4b1525ca-6a9a-4093-b5d2-321dc5080191 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b170acb342a24b58a044c78231fe52ac'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:16.021581 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4b1525ca-6a9a-4093-b5d2-321dc5080191 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b170acb342a24b58a044c78231fe52ac'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:16.023936 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4b1525ca-6a9a-4093-b5d2-321dc5080191 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b170acb342a24b58a044c78231fe52ac'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:16.041219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-10ad327a-e076-4d0e-b8a4-d2bbd7bdd619 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48956e00e2cd441ea7e6ce7c30dacf0e', 'request_id': 'req-10ad327a-e076-4d0e-b8a4-d2bbd7bdd619'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:16.052532 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4b1525ca-6a9a-4093-b5d2-321dc5080191 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b170acb342a24b58a044c78231fe52ac'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:16.055755 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4b1525ca-6a9a-4093-b5d2-321dc5080191 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b170acb342a24b58a044c78231fe52ac'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:16.083078 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-10ad327a-e076-4d0e-b8a4-d2bbd7bdd619 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48956e00e2cd441ea7e6ce7c30dacf0e', 'request_id': 'req-10ad327a-e076-4d0e-b8a4-d2bbd7bdd619'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:16.097592 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 903/1782] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:15 2022] DELETE /identity/v3/users/b170acb342a24b58a044c78231fe52ac => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:16.106069 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995946a-469a-4a31-85dc-703903577ad9 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:16.108262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995946a-469a-4a31-85dc-703903577ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:16.108868 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995946a-469a-4a31-85dc-703903577ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:16.112773 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-10ad327a-e076-4d0e-b8a4-d2bbd7bdd619 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48956e00e2cd441ea7e6ce7c30dacf0e', 'request_id': 'req-10ad327a-e076-4d0e-b8a4-d2bbd7bdd619'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:16.121477 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:16.139293 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-10ad327a-e076-4d0e-b8a4-d2bbd7bdd619 admin admin] The token cache is being invalidate because project 48956e00e2cd441ea7e6ce7c30dacf0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:16.139885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-10ad327a-e076-4d0e-b8a4-d2bbd7bdd619 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:16.141999 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-10ad327a-e076-4d0e-b8a4-d2bbd7bdd619 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:16.150141 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 880/1783] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:15 2022] DELETE /identity/v3/projects/48956e00e2cd441ea7e6ce7c30dacf0e => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:39:16.150141 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32093e-0113-4be4-b50c-12f317936b9c None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:16.150141 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32093e-0113-4be4-b50c-12f317936b9c None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:16.150141 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32093e-0113-4be4-b50c-12f317936b9c None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:16.155543 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 881/1784] 10.222.0.47 () {54 vars in 912 bytes} [Thu Aug 4 04:39:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:39:16.158318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4a94e-6932-4a74-83ce-5c1803dc5f3c None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:16.158603 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4a94e-6932-4a74-83ce-5c1803dc5f3c None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:16.158892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4a94e-6932-4a74-83ce-5c1803dc5f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:16.224994 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-6995946a-469a-4a31-85dc-703903577ad9 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:16.226763 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-e4f4a94e-6932-4a74-83ce-5c1803dc5f3c None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:16.228001 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-e4f4a94e-6932-4a74-83ce-5c1803dc5f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:16.252721 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6995946a-469a-4a31-85dc-703903577ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:16.389618 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 882/1785] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:16.389618 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a3043-7322-4996-93de-a75d3e16ab49 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:16.389618 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a3043-7322-4996-93de-a75d3e16ab49 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:16.389975 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a3043-7322-4996-93de-a75d3e16ab49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:16.421696 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-281a3043-7322-4996-93de-a75d3e16ab49 None None] MFA Rules not processed for user `1a2a647f97fc4d369f562e8b5a4bf68c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:16.421696 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-281a3043-7322-4996-93de-a75d3e16ab49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:16.466882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 904/1786] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:16.478669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b93f4-bc63-40c4-9815-08121048d197 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:16.481696 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b93f4-bc63-40c4-9815-08121048d197 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:16.486859 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b93f4-bc63-40c4-9815-08121048d197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:16.553145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-497b93f4-bc63-40c4-9815-08121048d197 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:16.557767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-497b93f4-bc63-40c4-9815-08121048d197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:16.559083 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 883/1787] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Aug 4 04:39:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:16.563537 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a3043-7322-4996-93de-a75d3e16ab49 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:16.566647 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-281a3043-7322-4996-93de-a75d3e16ab49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:16.594117 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 905/1788] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:16.594117 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b93f4-bc63-40c4-9815-08121048d197 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:16.602223 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-497b93f4-bc63-40c4-9815-08121048d197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:16.617443 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a3043-7322-4996-93de-a75d3e16ab49 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:16.648020 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b93f4-bc63-40c4-9815-08121048d197 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:16.648858 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b463fb4-280d-405a-99c5-f601c6f899ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:16.649683 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b463fb4-280d-405a-99c5-f601c6f899ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:16.650729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b463fb4-280d-405a-99c5-f601c6f899ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:16.651543 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b463fb4-280d-405a-99c5-f601c6f899ef admin admin] PATH_INFO: `/v3/users/dfe222e6e5f84b70845df382889a14f5` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:16.654962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b463fb4-280d-405a-99c5-f601c6f899ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfe222e6e5f84b70845df382889a14f5)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:16.658096 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e2cc7-1901-4f8f-a39a-6d0c23466021 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:16.660517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e2cc7-1901-4f8f-a39a-6d0c23466021 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:16.661390 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b463fb4-280d-405a-99c5-f601c6f899ef admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:16.661757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e2cc7-1901-4f8f-a39a-6d0c23466021 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:16.662139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e2cc7-1901-4f8f-a39a-6d0c23466021 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:16.688660 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e2cc7-1901-4f8f-a39a-6d0c23466021 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:16.692281 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e2cc7-1901-4f8f-a39a-6d0c23466021 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:16.733576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 906/1789] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:16.735247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e2cc7-1901-4f8f-a39a-6d0c23466021 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:16.738680 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c69e2cc7-1901-4f8f-a39a-6d0c23466021 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:16.782351 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e2cc7-1901-4f8f-a39a-6d0c23466021 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:16.797677 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6f617-9126-4c17-8854-e364a1dc2369 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:16.800677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9b463fb4-280d-405a-99c5-f601c6f899ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfe222e6e5f84b70845df382889a14f5'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:16.801895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6f617-9126-4c17-8854-e364a1dc2369 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:16.803087 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9b463fb4-280d-405a-99c5-f601c6f899ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfe222e6e5f84b70845df382889a14f5'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:16.804551 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6f617-9126-4c17-8854-e364a1dc2369 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:16.804551 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6f617-9126-4c17-8854-e364a1dc2369 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:16.813681 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c9f6f617-9126-4c17-8854-e364a1dc2369 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:16.859629 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6f617-9126-4c17-8854-e364a1dc2369 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:16.869618 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6f617-9126-4c17-8854-e364a1dc2369 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:16.876925 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9b463fb4-280d-405a-99c5-f601c6f899ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfe222e6e5f84b70845df382889a14f5'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:16.901833 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9b463fb4-280d-405a-99c5-f601c6f899ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfe222e6e5f84b70845df382889a14f5'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:16.903276 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9b463fb4-280d-405a-99c5-f601c6f899ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfe222e6e5f84b70845df382889a14f5'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:16.912121 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 907/1790] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:16.915249 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6f617-9126-4c17-8854-e364a1dc2369 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:16.920804 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c9f6f617-9126-4c17-8854-e364a1dc2369 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:16.961017 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 884/1791] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:16 2022] DELETE /identity/v3/users/dfe222e6e5f84b70845df382889a14f5 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:16.965653 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404dddcd-8c1d-48ad-86af-194bdb1ca84c None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:16.966290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404dddcd-8c1d-48ad-86af-194bdb1ca84c None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:16.967187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404dddcd-8c1d-48ad-86af-194bdb1ca84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:17.021896 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-404dddcd-8c1d-48ad-86af-194bdb1ca84c None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:17.022343 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6f617-9126-4c17-8854-e364a1dc2369 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:17.025435 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-404dddcd-8c1d-48ad-86af-194bdb1ca84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:17.043865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:17.069612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:17.244189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 885/1792] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:17.260714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404dddcd-8c1d-48ad-86af-194bdb1ca84c None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:17.263038 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-404dddcd-8c1d-48ad-86af-194bdb1ca84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:17.279661 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240a677-8607-4cdd-83b3-28bf8adb5eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:17.280820 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240a677-8607-4cdd-83b3-28bf8adb5eee admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:17.281136 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240a677-8607-4cdd-83b3-28bf8adb5eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:17.281435 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240a677-8607-4cdd-83b3-28bf8adb5eee admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:17.282561 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240a677-8607-4cdd-83b3-28bf8adb5eee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:39:17.282892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240a677-8607-4cdd-83b3-28bf8adb5eee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:17.285733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240a677-8607-4cdd-83b3-28bf8adb5eee admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:17.315776 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 908/1793] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:39:17.317032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240a677-8607-4cdd-83b3-28bf8adb5eee admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:17.341529 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240a677-8607-4cdd-83b3-28bf8adb5eee admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:17.350724 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15dc00-1fec-487f-b812-2a30c3be2a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:17.351479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15dc00-1fec-487f-b812-2a30c3be2a8c admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:17.351768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15dc00-1fec-487f-b812-2a30c3be2a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:17.352166 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15dc00-1fec-487f-b812-2a30c3be2a8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:17.352871 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15dc00-1fec-487f-b812-2a30c3be2a8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:17.355264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15dc00-1fec-487f-b812-2a30c3be2a8c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:17.410792 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404dddcd-8c1d-48ad-86af-194bdb1ca84c None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:17.411179 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 909/1794] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:17 2022] POST /identity/v3/projects => generated 371 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:17.416978 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15dc00-1fec-487f-b812-2a30c3be2a8c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:17.605712 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4efee7-bde6-4b00-bf7c-d8996766e2d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:17.606861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4efee7-bde6-4b00-bf7c-d8996766e2d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:17.607030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4efee7-bde6-4b00-bf7c-d8996766e2d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:17.608827 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4efee7-bde6-4b00-bf7c-d8996766e2d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:17.613049 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15dc00-1fec-487f-b812-2a30c3be2a8c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:17.627584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2e6c9-7cfb-4992-8a43-e5717dc1c403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:17.628539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2e6c9-7cfb-4992-8a43-e5717dc1c403 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:17.628886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2e6c9-7cfb-4992-8a43-e5717dc1c403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:17.629330 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2e6c9-7cfb-4992-8a43-e5717dc1c403 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:17.630807 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2e6c9-7cfb-4992-8a43-e5717dc1c403 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:17.635297 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2e6c9-7cfb-4992-8a43-e5717dc1c403 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:17.644252 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4efee7-bde6-4b00-bf7c-d8996766e2d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:17.645746 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4efee7-bde6-4b00-bf7c-d8996766e2d2 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:17.684549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 910/1795] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:17 2022] POST /identity/v3/users => generated 286 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:17.688096 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2e6c9-7cfb-4992-8a43-e5717dc1c403 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:17.690482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b5f2e6c9-7cfb-4992-8a43-e5717dc1c403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:17.717326 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2e6c9-7cfb-4992-8a43-e5717dc1c403 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:17.739313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615bc842-cad4-4ec1-afb1-7b840c64455b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:17.739608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 886/1796] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:39:17.740403 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615bc842-cad4-4ec1-afb1-7b840c64455b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:17.740865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615bc842-cad4-4ec1-afb1-7b840c64455b admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:17.740993 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4efee7-bde6-4b00-bf7c-d8996766e2d2 service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:17.741309 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615bc842-cad4-4ec1-afb1-7b840c64455b admin admin] PATH_INFO: `/v3/projects/56d36a2533754402857d34d87b48e032` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:17.742809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615bc842-cad4-4ec1-afb1-7b840c64455b admin admin] RBAC: Authorizing `identity:delete_project(project_id=56d36a2533754402857d34d87b48e032)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:17.744843 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615bc842-cad4-4ec1-afb1-7b840c64455b admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:17.762822 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4efee7-bde6-4b00-bf7c-d8996766e2d2 service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:17.779330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e9f79-2b54-44e6-a4aa-6a0ac83d1f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:17.797727 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e9f79-2b54-44e6-a4aa-6a0ac83d1f80 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:17.799838 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e9f79-2b54-44e6-a4aa-6a0ac83d1f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:17.800360 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e9f79-2b54-44e6-a4aa-6a0ac83d1f80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:17.801453 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160e9f79-2b54-44e6-a4aa-6a0ac83d1f80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:17.803749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160e9f79-2b54-44e6-a4aa-6a0ac83d1f80 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:17.844346 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 887/1797] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:17.859400 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e9f79-2b54-44e6-a4aa-6a0ac83d1f80 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:17.886167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e9f79-2b54-44e6-a4aa-6a0ac83d1f80 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:17.893164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-615bc842-cad4-4ec1-afb1-7b840c64455b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56d36a2533754402857d34d87b48e032', 'request_id': 'req-615bc842-cad4-4ec1-afb1-7b840c64455b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:17.915737 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-615bc842-cad4-4ec1-afb1-7b840c64455b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56d36a2533754402857d34d87b48e032', 'request_id': 'req-615bc842-cad4-4ec1-afb1-7b840c64455b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:17.940596 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-615bc842-cad4-4ec1-afb1-7b840c64455b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56d36a2533754402857d34d87b48e032', 'request_id': 'req-615bc842-cad4-4ec1-afb1-7b840c64455b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:17.954588 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:18.032941 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-615bc842-cad4-4ec1-afb1-7b840c64455b admin admin] The token cache is being invalidate because project 56d36a2533754402857d34d87b48e032 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:18.033421 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-615bc842-cad4-4ec1-afb1-7b840c64455b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:18.037686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-615bc842-cad4-4ec1-afb1-7b840c64455b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:18.042888 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 911/1798] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:17 2022] DELETE /identity/v3/projects/56d36a2533754402857d34d87b48e032 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:18.061731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615bc842-cad4-4ec1-afb1-7b840c64455b admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:18.062994 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf952409-aad7-419f-88d4-9b65aacf8421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:18.064172 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf952409-aad7-419f-88d4-9b65aacf8421 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:18.069991 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf952409-aad7-419f-88d4-9b65aacf8421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:18.070823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-615bc842-cad4-4ec1-afb1-7b840c64455b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:18.071493 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf952409-aad7-419f-88d4-9b65aacf8421 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:18.074342 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf952409-aad7-419f-88d4-9b65aacf8421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:18.086956 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf952409-aad7-419f-88d4-9b65aacf8421 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:18.115915 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615bc842-cad4-4ec1-afb1-7b840c64455b admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:18.150207 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aadac6-645e-42e8-a507-60958f46cef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:18.150920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aadac6-645e-42e8-a507-60958f46cef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:18.151211 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aadac6-645e-42e8-a507-60958f46cef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:18.151418 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aadac6-645e-42e8-a507-60958f46cef4 admin admin] PATH_INFO: `/v3/users/a1b2c14ca679487bb3636b69db47e2d6` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:18.156374 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 888/1799] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:18.160262 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf952409-aad7-419f-88d4-9b65aacf8421 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:18.167508 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aadac6-645e-42e8-a507-60958f46cef4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1b2c14ca679487bb3636b69db47e2d6)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:18.170683 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-bf952409-aad7-419f-88d4-9b65aacf8421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:18.176067 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aadac6-645e-42e8-a507-60958f46cef4 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:18.201366 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:18.218854 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf952409-aad7-419f-88d4-9b65aacf8421 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:18.234286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf1f5c-4591-4d91-a4dd-5c70732f4a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:18.235114 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf1f5c-4591-4d91-a4dd-5c70732f4a38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:18.236564 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf1f5c-4591-4d91-a4dd-5c70732f4a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:18.237052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf1f5c-4591-4d91-a4dd-5c70732f4a38 admin admin] PATH_INFO: `/v3/projects/6fa49c2084db42859cab0a2fbe5454db/users/a13a0b07b3394507b7dc4849f0f0eaa1/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:18.319336 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf1f5c-4591-4d91-a4dd-5c70732f4a38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fa49c2084db42859cab0a2fbe5454db, user_id=a13a0b07b3394507b7dc4849f0f0eaa1, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:18.343746 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf1f5c-4591-4d91-a4dd-5c70732f4a38 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:18.400683 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 889/1800] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:18 2022] PUT /identity/v3/projects/6fa49c2084db42859cab0a2fbe5454db/users/a13a0b07b3394507b7dc4849f0f0eaa1/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:18.420855 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c17b0b-a9ef-4fd3-8e58-4a17f5c7143d None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:18.421174 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c17b0b-a9ef-4fd3-8e58-4a17f5c7143d None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:18.421627 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c17b0b-a9ef-4fd3-8e58-4a17f5c7143d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:18.431919 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e2aadac6-645e-42e8-a507-60958f46cef4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1b2c14ca679487bb3636b69db47e2d6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:18.459477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e2aadac6-645e-42e8-a507-60958f46cef4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1b2c14ca679487bb3636b69db47e2d6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:18.460290 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e2aadac6-645e-42e8-a507-60958f46cef4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1b2c14ca679487bb3636b69db47e2d6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:18.478146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e2aadac6-645e-42e8-a507-60958f46cef4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1b2c14ca679487bb3636b69db47e2d6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:18.479158 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e2aadac6-645e-42e8-a507-60958f46cef4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1b2c14ca679487bb3636b69db47e2d6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:18.563628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 912/1801] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:18 2022] DELETE /identity/v3/users/a1b2c14ca679487bb3636b69db47e2d6 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:18.563946 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aadac6-645e-42e8-a507-60958f46cef4 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:18.564928 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-c7c17b0b-a9ef-4fd3-8e58-4a17f5c7143d None None] MFA Rules not processed for user `a13a0b07b3394507b7dc4849f0f0eaa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:18.566656 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c7c17b0b-a9ef-4fd3-8e58-4a17f5c7143d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:18.568851 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e2aadac6-645e-42e8-a507-60958f46cef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:18.918914 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aadac6-645e-42e8-a507-60958f46cef4 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:18.962752 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44440af0-d661-49c5-a702-52867af7ec1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:18.963549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44440af0-d661-49c5-a702-52867af7ec1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:18.963913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44440af0-d661-49c5-a702-52867af7ec1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:18.964250 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44440af0-d661-49c5-a702-52867af7ec1d admin admin] PATH_INFO: `/v3/projects/9fcc6257d02c4927b30006a3a6c980ae` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:18.971363 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44440af0-d661-49c5-a702-52867af7ec1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fcc6257d02c4927b30006a3a6c980ae)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:18.984893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44440af0-d661-49c5-a702-52867af7ec1d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:19.101774 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 890/1802] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 684 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:19.109509 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f487b26-3713-4313-96f6-edb8a8b00226 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:19.111067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f487b26-3713-4313-96f6-edb8a8b00226 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:19.111374 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f487b26-3713-4313-96f6-edb8a8b00226 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:19.119582 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 891/1803] 10.222.0.47 () {54 vars in 921 bytes} [Thu Aug 4 04:39:19 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:39:19.124493 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f487b26-3713-4313-96f6-edb8a8b00226 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:19.130444 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-3f487b26-3713-4313-96f6-edb8a8b00226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:19.154534 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-44440af0-d661-49c5-a702-52867af7ec1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fcc6257d02c4927b30006a3a6c980ae', 'request_id': 'req-44440af0-d661-49c5-a702-52867af7ec1d'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:19.171513 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-44440af0-d661-49c5-a702-52867af7ec1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fcc6257d02c4927b30006a3a6c980ae', 'request_id': 'req-44440af0-d661-49c5-a702-52867af7ec1d'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:19.179426 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-44440af0-d661-49c5-a702-52867af7ec1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fcc6257d02c4927b30006a3a6c980ae', 'request_id': 'req-44440af0-d661-49c5-a702-52867af7ec1d'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:19.182739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:19.207123 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-44440af0-d661-49c5-a702-52867af7ec1d admin admin] The token cache is being invalidate because project 9fcc6257d02c4927b30006a3a6c980ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:19.207508 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-44440af0-d661-49c5-a702-52867af7ec1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:19.209511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-44440af0-d661-49c5-a702-52867af7ec1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:19.218067 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 913/1804] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:18 2022] DELETE /identity/v3/projects/9fcc6257d02c4927b30006a3a6c980ae => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:19.218785 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65641dfd-f830-4e6d-8f0e-21e6ef84826f None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:19.219535 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65641dfd-f830-4e6d-8f0e-21e6ef84826f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:19.220541 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65641dfd-f830-4e6d-8f0e-21e6ef84826f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:19.308435 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-65641dfd-f830-4e6d-8f0e-21e6ef84826f None None] MFA Rules not processed for user `caa2901451aa4554bdfcac1bddbc620a`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:19.309247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-65641dfd-f830-4e6d-8f0e-21e6ef84826f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:19.391224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f487b26-3713-4313-96f6-edb8a8b00226 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:19.427303 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a889738-cae8-420b-a16d-8c20d7fc1f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:19.428160 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a889738-cae8-420b-a16d-8c20d7fc1f0c admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:19.428657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a889738-cae8-420b-a16d-8c20d7fc1f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:19.429378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a889738-cae8-420b-a16d-8c20d7fc1f0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:19.430333 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a889738-cae8-420b-a16d-8c20d7fc1f0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:19.434141 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a889738-cae8-420b-a16d-8c20d7fc1f0c admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:19.520191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 892/1805] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:19 2022] POST /identity/v3/projects => generated 373 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:19.524368 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a889738-cae8-420b-a16d-8c20d7fc1f0c admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:19.528776 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 914/1806] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Aug 4 04:39:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:19.529671 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-2a889738-cae8-420b-a16d-8c20d7fc1f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:19.530686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65641dfd-f830-4e6d-8f0e-21e6ef84826f None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:19.544842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-65641dfd-f830-4e6d-8f0e-21e6ef84826f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:19.573270 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65641dfd-f830-4e6d-8f0e-21e6ef84826f None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:19.672135 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c45000-0855-4e7f-8f2f-8e0384978871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:19.672790 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c45000-0855-4e7f-8f2f-8e0384978871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:19.673072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c45000-0855-4e7f-8f2f-8e0384978871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:19.673341 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c45000-0855-4e7f-8f2f-8e0384978871 admin admin] PATH_INFO: `/v3/projects/926952a897c8470fb3ef11180dbea64f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:19.674780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c45000-0855-4e7f-8f2f-8e0384978871 admin admin] RBAC: Authorizing `identity:delete_project(project_id=926952a897c8470fb3ef11180dbea64f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:19.677290 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c45000-0855-4e7f-8f2f-8e0384978871 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:19.681930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:19.706984 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a889738-cae8-420b-a16d-8c20d7fc1f0c admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:19.713261 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813828b2-8719-4146-83ae-474d5d651010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:19.714238 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813828b2-8719-4146-83ae-474d5d651010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:19.714739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813828b2-8719-4146-83ae-474d5d651010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:19.715154 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813828b2-8719-4146-83ae-474d5d651010 admin admin] PATH_INFO: `/v3/users/1b8364515e224123905eb48442fc516f` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:19.720088 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813828b2-8719-4146-83ae-474d5d651010 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b8364515e224123905eb48442fc516f)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:19.721475 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813828b2-8719-4146-83ae-474d5d651010 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:19.771063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b5c45000-0855-4e7f-8f2f-8e0384978871 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '926952a897c8470fb3ef11180dbea64f', 'request_id': 'req-b5c45000-0855-4e7f-8f2f-8e0384978871'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:19.796413 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b5c45000-0855-4e7f-8f2f-8e0384978871 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '926952a897c8470fb3ef11180dbea64f', 'request_id': 'req-b5c45000-0855-4e7f-8f2f-8e0384978871'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:19.804410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b5c45000-0855-4e7f-8f2f-8e0384978871 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '926952a897c8470fb3ef11180dbea64f', 'request_id': 'req-b5c45000-0855-4e7f-8f2f-8e0384978871'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:19.828663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b5c45000-0855-4e7f-8f2f-8e0384978871 admin admin] The token cache is being invalidate because project 926952a897c8470fb3ef11180dbea64f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:19.829126 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b5c45000-0855-4e7f-8f2f-8e0384978871 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:19.833110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b5c45000-0855-4e7f-8f2f-8e0384978871 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:19.833394 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-813828b2-8719-4146-83ae-474d5d651010 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b8364515e224123905eb48442fc516f'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:19.836930 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 915/1807] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:19 2022] DELETE /identity/v3/projects/926952a897c8470fb3ef11180dbea64f => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:19.837593 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-813828b2-8719-4146-83ae-474d5d651010 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b8364515e224123905eb48442fc516f'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:19.838750 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c45000-0855-4e7f-8f2f-8e0384978871 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:19.858235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b5c45000-0855-4e7f-8f2f-8e0384978871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:19.878697 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-813828b2-8719-4146-83ae-474d5d651010 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b8364515e224123905eb48442fc516f'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:19.896029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-813828b2-8719-4146-83ae-474d5d651010 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b8364515e224123905eb48442fc516f'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:19.898380 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-813828b2-8719-4146-83ae-474d5d651010 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b8364515e224123905eb48442fc516f'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:19.918846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:19.920307 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 893/1808] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:19 2022] DELETE /identity/v3/users/1b8364515e224123905eb48442fc516f => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:19.921646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813828b2-8719-4146-83ae-474d5d651010 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:19.938804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-813828b2-8719-4146-83ae-474d5d651010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:19.987322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c45000-0855-4e7f-8f2f-8e0384978871 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:20.075473 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813828b2-8719-4146-83ae-474d5d651010 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:20.083609 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bcd64-4504-4441-b6f7-ebd837fdfef5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:20.084296 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bcd64-4504-4441-b6f7-ebd837fdfef5 service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:20.084579 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bcd64-4504-4441-b6f7-ebd837fdfef5 service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:20.084853 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bcd64-4504-4441-b6f7-ebd837fdfef5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:20.086830 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-239bcd64-4504-4441-b6f7-ebd837fdfef5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:20.094316 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615aac4a-5356-4893-a3fc-30e631daec9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:20.095058 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615aac4a-5356-4893-a3fc-30e631daec9c admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:20.095436 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615aac4a-5356-4893-a3fc-30e631daec9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:20.095803 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615aac4a-5356-4893-a3fc-30e631daec9c admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:20.096569 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615aac4a-5356-4893-a3fc-30e631daec9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:20.098616 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615aac4a-5356-4893-a3fc-30e631daec9c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:20.108181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bcd64-4504-4441-b6f7-ebd837fdfef5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:20.112005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bcd64-4504-4441-b6f7-ebd837fdfef5 service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:20.125279 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 916/1809] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:19 2022] POST /identity/v3/users => generated 293 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:20.125799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 894/1810] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:20.126874 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1fbcc0-f8ef-4907-bebd-21a11df1e0d7 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:20.127308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1fbcc0-f8ef-4907-bebd-21a11df1e0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:20.127634 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1fbcc0-f8ef-4907-bebd-21a11df1e0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:20.130350 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bcd64-4504-4441-b6f7-ebd837fdfef5 service nova] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:20.132087 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-239bcd64-4504-4441-b6f7-ebd837fdfef5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:20.155649 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bcd64-4504-4441-b6f7-ebd837fdfef5 service nova] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:20.159574 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-9a1fbcc0-f8ef-4907-bebd-21a11df1e0d7 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:20.160802 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9a1fbcc0-f8ef-4907-bebd-21a11df1e0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:20.161196 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa41362-ad2a-440d-b4e8-fd68e838cfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:20.161999 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa41362-ad2a-440d-b4e8-fd68e838cfc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:20.162462 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa41362-ad2a-440d-b4e8-fd68e838cfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:20.162613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa41362-ad2a-440d-b4e8-fd68e838cfc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:20.164468 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa41362-ad2a-440d-b4e8-fd68e838cfc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:20.167451 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa41362-ad2a-440d-b4e8-fd68e838cfc5 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:20.172999 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 917/1811] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:20.180679 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc00613-3e7e-4cb2-afe6-1b96fa1b9aa4 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:20.180960 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc00613-3e7e-4cb2-afe6-1b96fa1b9aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:20.181234 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc00613-3e7e-4cb2-afe6-1b96fa1b9aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:20.185628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 895/1812] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 04:39:20.190596 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa41362-ad2a-440d-b4e8-fd68e838cfc5 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:20.207083 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa41362-ad2a-440d-b4e8-fd68e838cfc5 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:20.210198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-8cc00613-3e7e-4cb2-afe6-1b96fa1b9aa4 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:20.211419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8cc00613-3e7e-4cb2-afe6-1b96fa1b9aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:20.215624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa547b9-387c-4043-af38-86fb098586ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:20.221161 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa547b9-387c-4043-af38-86fb098586ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:20.222502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa547b9-387c-4043-af38-86fb098586ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:20.222803 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa547b9-387c-4043-af38-86fb098586ef admin admin] PATH_INFO: `/v3/projects/8a9a5b74819b4bda926c9f5345ee14d0/users/b3de9bce01c94017b921b6ddba404854/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:20.226923 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 918/1813] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:39:20.227863 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc00613-3e7e-4cb2-afe6-1b96fa1b9aa4 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:20.265362 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc00613-3e7e-4cb2-afe6-1b96fa1b9aa4 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:20.277041 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd018f52-3211-43ba-9fe6-347a6019e54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:20.280616 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa547b9-387c-4043-af38-86fb098586ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a9a5b74819b4bda926c9f5345ee14d0, user_id=b3de9bce01c94017b921b6ddba404854, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:20.283739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd018f52-3211-43ba-9fe6-347a6019e54f admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:20.284036 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd018f52-3211-43ba-9fe6-347a6019e54f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:20.284387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd018f52-3211-43ba-9fe6-347a6019e54f admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:20.288360 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd018f52-3211-43ba-9fe6-347a6019e54f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:39:20.288360 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd018f52-3211-43ba-9fe6-347a6019e54f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:20.290053 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd018f52-3211-43ba-9fe6-347a6019e54f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:20.293119 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa547b9-387c-4043-af38-86fb098586ef admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:20.320293 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 919/1814] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:39:20.325062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 896/1815] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:20 2022] PUT /identity/v3/projects/8a9a5b74819b4bda926c9f5345ee14d0/users/b3de9bce01c94017b921b6ddba404854/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:20.327285 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd018f52-3211-43ba-9fe6-347a6019e54f admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:20.328172 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa547b9-387c-4043-af38-86fb098586ef admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:20.372201 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd018f52-3211-43ba-9fe6-347a6019e54f admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:20.385509 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:20.461208 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa547b9-387c-4043-af38-86fb098586ef admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:20.483998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69ed6f-6ac1-482c-86a6-1bb6e2a8c47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:20.484779 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69ed6f-6ac1-482c-86a6-1bb6e2a8c47e admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:20.485061 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69ed6f-6ac1-482c-86a6-1bb6e2a8c47e admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:20.485690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69ed6f-6ac1-482c-86a6-1bb6e2a8c47e admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:20.498316 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69ed6f-6ac1-482c-86a6-1bb6e2a8c47e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:20.504195 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb4a9f-eb2c-4c5c-aad6-c86880283f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:20.506656 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69ed6f-6ac1-482c-86a6-1bb6e2a8c47e admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:20.507744 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb4a9f-eb2c-4c5c-aad6-c86880283f82 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:20.508703 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb4a9f-eb2c-4c5c-aad6-c86880283f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:20.509821 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb4a9f-eb2c-4c5c-aad6-c86880283f82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:20.511514 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb4a9f-eb2c-4c5c-aad6-c86880283f82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:20.514671 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb4a9f-eb2c-4c5c-aad6-c86880283f82 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:20.533601 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 897/1816] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:20.533601 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69ed6f-6ac1-482c-86a6-1bb6e2a8c47e admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:20.534309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-fa69ed6f-6ac1-482c-86a6-1bb6e2a8c47e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:20.562212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69ed6f-6ac1-482c-86a6-1bb6e2a8c47e admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:20.566964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 920/1817] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:20 2022] POST /identity/v3/projects => generated 381 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:20.571116 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb4a9f-eb2c-4c5c-aad6-c86880283f82 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:20.572897 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a12ab0-146c-40dd-91d6-c615f145b3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:20.573603 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a12ab0-146c-40dd-91d6-c615f145b3d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:20.573965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a12ab0-146c-40dd-91d6-c615f145b3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:20.574230 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a12ab0-146c-40dd-91d6-c615f145b3d0 admin admin] PATH_INFO: `/v3/projects/00207df8167a4333af912e59be49b9a6` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:20.576376 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a12ab0-146c-40dd-91d6-c615f145b3d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00207df8167a4333af912e59be49b9a6)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:20.579223 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a12ab0-146c-40dd-91d6-c615f145b3d0 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:20.618094 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:20.646349 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-00a12ab0-146c-40dd-91d6-c615f145b3d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00207df8167a4333af912e59be49b9a6', 'request_id': 'req-00a12ab0-146c-40dd-91d6-c615f145b3d0'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:20.663716 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-00a12ab0-146c-40dd-91d6-c615f145b3d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00207df8167a4333af912e59be49b9a6', 'request_id': 'req-00a12ab0-146c-40dd-91d6-c615f145b3d0'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:20.675489 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-00a12ab0-146c-40dd-91d6-c615f145b3d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00207df8167a4333af912e59be49b9a6', 'request_id': 'req-00a12ab0-146c-40dd-91d6-c615f145b3d0'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:20.699603 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-00a12ab0-146c-40dd-91d6-c615f145b3d0 admin admin] The token cache is being invalidate because project 00207df8167a4333af912e59be49b9a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:20.699895 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-00a12ab0-146c-40dd-91d6-c615f145b3d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:20.701568 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-00a12ab0-146c-40dd-91d6-c615f145b3d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:20.703389 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 898/1818] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:20 2022] DELETE /identity/v3/projects/00207df8167a4333af912e59be49b9a6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:20.704171 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a12ab0-146c-40dd-91d6-c615f145b3d0 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:20.705772 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-00a12ab0-146c-40dd-91d6-c615f145b3d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:20.726440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb4a9f-eb2c-4c5c-aad6-c86880283f82 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:20.812980 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a12ab0-146c-40dd-91d6-c615f145b3d0 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:20.841111 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3aca4-d6f6-4b16-9919-3242f6b40072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:20.845610 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3aca4-d6f6-4b16-9919-3242f6b40072 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:20.845610 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3aca4-d6f6-4b16-9919-3242f6b40072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:20.853103 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3aca4-d6f6-4b16-9919-3242f6b40072 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:20.854683 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3aca4-d6f6-4b16-9919-3242f6b40072 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:20.860351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3aca4-d6f6-4b16-9919-3242f6b40072 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:20.867010 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dcdf35-b19f-43a2-80f9-680e6a6d4de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:20.867809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcdf35-b19f-43a2-80f9-680e6a6d4de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:20.868163 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcdf35-b19f-43a2-80f9-680e6a6d4de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:20.868504 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcdf35-b19f-43a2-80f9-680e6a6d4de5 admin admin] PATH_INFO: `/v3/projects/8a9a5b74819b4bda926c9f5345ee14d0/users/b3de9bce01c94017b921b6ddba404854/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:20.870707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dcdf35-b19f-43a2-80f9-680e6a6d4de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a9a5b74819b4bda926c9f5345ee14d0, user_id=b3de9bce01c94017b921b6ddba404854, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:20.875319 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dcdf35-b19f-43a2-80f9-680e6a6d4de5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:20.895387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 921/1819] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:20 2022] PUT /identity/v3/projects/8a9a5b74819b4bda926c9f5345ee14d0/users/b3de9bce01c94017b921b6ddba404854/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:20.897430 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9816bb-f162-4e52-8c7c-8cc9c9e7dc4f None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:20.898026 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9816bb-f162-4e52-8c7c-8cc9c9e7dc4f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:20.898415 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 899/1820] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:20 2022] POST /identity/v3/users => generated 291 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:20.898926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9816bb-f162-4e52-8c7c-8cc9c9e7dc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:20.902617 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3aca4-d6f6-4b16-9919-3242f6b40072 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:20.926565 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3aca4-d6f6-4b16-9919-3242f6b40072 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:20.931346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-9e9816bb-f162-4e52-8c7c-8cc9c9e7dc4f None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:20.932937 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9e9816bb-f162-4e52-8c7c-8cc9c9e7dc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:20.986965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:21.061316 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 922/1821] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:21.061316 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ed9c7-3897-4c80-b70d-476c04b18bc6 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:21.061316 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ed9c7-3897-4c80-b70d-476c04b18bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:21.061316 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ed9c7-3897-4c80-b70d-476c04b18bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:21.080551 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4770c123-3469-44f7-8d00-e3e839b0c38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:21.081418 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770c123-3469-44f7-8d00-e3e839b0c38e admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:21.081726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770c123-3469-44f7-8d00-e3e839b0c38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:21.082001 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770c123-3469-44f7-8d00-e3e839b0c38e admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:21.083983 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4770c123-3469-44f7-8d00-e3e839b0c38e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:21.088219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4770c123-3469-44f7-8d00-e3e839b0c38e admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:21.129446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 900/1822] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:21.131141 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550b5fe-722b-4b60-8708-8fd8a05bf3b3 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:21.131437 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550b5fe-722b-4b60-8708-8fd8a05bf3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:21.131715 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550b5fe-722b-4b60-8708-8fd8a05bf3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:21.145836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-472ed9c7-3897-4c80-b70d-476c04b18bc6 None None] MFA Rules not processed for user `b3de9bce01c94017b921b6ddba404854`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:21.147579 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-472ed9c7-3897-4c80-b70d-476c04b18bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:21.164152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-7550b5fe-722b-4b60-8708-8fd8a05bf3b3 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:21.167065 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-7550b5fe-722b-4b60-8708-8fd8a05bf3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:21.189187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 901/1823] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:21.189187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550b5fe-722b-4b60-8708-8fd8a05bf3b3 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:21.208919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550b5fe-722b-4b60-8708-8fd8a05bf3b3 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:21.214567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c2264-0972-42b3-92cb-6c21ffd28cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:21.215343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c2264-0972-42b3-92cb-6c21ffd28cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:21.215641 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c2264-0972-42b3-92cb-6c21ffd28cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:21.215918 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c2264-0972-42b3-92cb-6c21ffd28cb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:21.216900 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5c2264-0972-42b3-92cb-6c21ffd28cb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:21.218596 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5c2264-0972-42b3-92cb-6c21ffd28cb4 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:21.229491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:21.230053 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 902/1824] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:21.230969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c2264-0972-42b3-92cb-6c21ffd28cb4 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:21.261322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c2264-0972-42b3-92cb-6c21ffd28cb4 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:21.268988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829e432-8d7e-49ed-be95-93d763ca2d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:21.270042 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829e432-8d7e-49ed-be95-93d763ca2d9c admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:21.270501 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829e432-8d7e-49ed-be95-93d763ca2d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:21.270939 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829e432-8d7e-49ed-be95-93d763ca2d9c admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:21.271915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829e432-8d7e-49ed-be95-93d763ca2d9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:39:21.272614 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829e432-8d7e-49ed-be95-93d763ca2d9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:21.275039 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829e432-8d7e-49ed-be95-93d763ca2d9c admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:21.281391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 923/1825] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:21.283068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ed9c7-3897-4c80-b70d-476c04b18bc6 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:21.298271 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 903/1826] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:39:21.299041 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829e432-8d7e-49ed-be95-93d763ca2d9c admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:21.300886 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d829e432-8d7e-49ed-be95-93d763ca2d9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:21.301499 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ed9c7-3897-4c80-b70d-476c04b18bc6 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:21.318975 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7846b2-7557-4568-a1bd-71e5cc8c4360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:21.320540 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7846b2-7557-4568-a1bd-71e5cc8c4360 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:21.321026 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7846b2-7557-4568-a1bd-71e5cc8c4360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:21.321528 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7846b2-7557-4568-a1bd-71e5cc8c4360 admin admin] PATH_INFO: `/v3/projects/bd796d58aa704ead805b4478cf16e4ee/users/5acfefd72ff947f1b40a926739419409/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:21.339603 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7846b2-7557-4568-a1bd-71e5cc8c4360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd796d58aa704ead805b4478cf16e4ee, user_id=5acfefd72ff947f1b40a926739419409, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:21.343230 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7846b2-7557-4568-a1bd-71e5cc8c4360 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:21.364499 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 924/1827] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:21 2022] PUT /identity/v3/projects/bd796d58aa704ead805b4478cf16e4ee/users/5acfefd72ff947f1b40a926739419409/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:21.365503 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7846b2-7557-4568-a1bd-71e5cc8c4360 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:21.422467 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829e432-8d7e-49ed-be95-93d763ca2d9c admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:21.481822 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7846b2-7557-4568-a1bd-71e5cc8c4360 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:21.491172 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c671a-f148-48c8-9777-eb21dd558e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:21.492950 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c671a-f148-48c8-9777-eb21dd558e30 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:21.493350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c671a-f148-48c8-9777-eb21dd558e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:21.493995 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c671a-f148-48c8-9777-eb21dd558e30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:21.494707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c671a-f148-48c8-9777-eb21dd558e30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:21.497359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c671a-f148-48c8-9777-eb21dd558e30 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:21.515056 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b699be-89d8-4a1f-b90b-6c024047dad3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:21.516121 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b699be-89d8-4a1f-b90b-6c024047dad3 service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:21.516505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b699be-89d8-4a1f-b90b-6c024047dad3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:21.516877 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b699be-89d8-4a1f-b90b-6c024047dad3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:21.543978 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 925/1828] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:21 2022] POST /identity/v3/projects => generated 365 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:21.543978 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73612d57-0a72-418f-b7fb-432f527f153d None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:21.543978 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73612d57-0a72-418f-b7fb-432f527f153d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:21.543978 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73612d57-0a72-418f-b7fb-432f527f153d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:21.584630 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-73612d57-0a72-418f-b7fb-432f527f153d None None] MFA Rules not processed for user `5acfefd72ff947f1b40a926739419409`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:21.586242 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-73612d57-0a72-418f-b7fb-432f527f153d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:21.609385 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b699be-89d8-4a1f-b90b-6c024047dad3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:21.611076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b699be-89d8-4a1f-b90b-6c024047dad3 service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:21.625870 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 904/1829] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:21.632363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b699be-89d8-4a1f-b90b-6c024047dad3 service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:21.646414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:21.655164 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b699be-89d8-4a1f-b90b-6c024047dad3 service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:21.728193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 926/1830] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:21.730056 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73612d57-0a72-418f-b7fb-432f527f153d None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:21.746146 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c819ff0-c991-4673-bf0d-63ce67aded6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:21.747444 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c819ff0-c991-4673-bf0d-63ce67aded6d admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:21.748232 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c819ff0-c991-4673-bf0d-63ce67aded6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:21.748596 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c819ff0-c991-4673-bf0d-63ce67aded6d admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:21.750294 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c819ff0-c991-4673-bf0d-63ce67aded6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:21.752801 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c819ff0-c991-4673-bf0d-63ce67aded6d admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:21.787479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:21.807938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 905/1831] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:21 2022] POST /identity/v3/users => generated 283 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:21.810248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c819ff0-c991-4673-bf0d-63ce67aded6d admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:21.834613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c819ff0-c991-4673-bf0d-63ce67aded6d admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:21.847020 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c1dd2-d618-4d15-80a7-1d3f94b29843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:21.847667 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c1dd2-d618-4d15-80a7-1d3f94b29843 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:21.847954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c1dd2-d618-4d15-80a7-1d3f94b29843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:21.848229 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c1dd2-d618-4d15-80a7-1d3f94b29843 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:21.848959 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c1dd2-d618-4d15-80a7-1d3f94b29843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:21.859055 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c1dd2-d618-4d15-80a7-1d3f94b29843 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:21.871946 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 906/1832] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:21.877333 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c1dd2-d618-4d15-80a7-1d3f94b29843 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:21.880854 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73612d57-0a72-418f-b7fb-432f527f153d None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:21.889258 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c1dd2-d618-4d15-80a7-1d3f94b29843 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:21.894528 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11e10a-bfc8-4514-862b-f128f4c31a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:21.895338 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11e10a-bfc8-4514-862b-f128f4c31a23 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:21.895643 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11e10a-bfc8-4514-862b-f128f4c31a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:21.895988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11e10a-bfc8-4514-862b-f128f4c31a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:21.896794 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11e10a-bfc8-4514-862b-f128f4c31a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:21.898619 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11e10a-bfc8-4514-862b-f128f4c31a23 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:21.902258 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6225d57-bfba-4c53-871e-d0e36927a8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:21.903038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6225d57-bfba-4c53-871e-d0e36927a8ac admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:21.903176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6225d57-bfba-4c53-871e-d0e36927a8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:21.903456 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6225d57-bfba-4c53-871e-d0e36927a8ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:21.905828 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6225d57-bfba-4c53-871e-d0e36927a8ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:21.907804 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6225d57-bfba-4c53-871e-d0e36927a8ac admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:21.915444 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 907/1833] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:21.917212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11e10a-bfc8-4514-862b-f128f4c31a23 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:21.932269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11e10a-bfc8-4514-862b-f128f4c31a23 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:21.944091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20289c-04bf-420b-ac4d-68a88fee0e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:21.944091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20289c-04bf-420b-ac4d-68a88fee0e18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:21.944091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20289c-04bf-420b-ac4d-68a88fee0e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:21.944091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20289c-04bf-420b-ac4d-68a88fee0e18 admin admin] PATH_INFO: `/v3/projects/d5d6cbe484a7475ea280b15b7a798789/users/89b164163ed842c1869a2233212a3aa2/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:21.981145 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20289c-04bf-420b-ac4d-68a88fee0e18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5d6cbe484a7475ea280b15b7a798789, user_id=89b164163ed842c1869a2233212a3aa2, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:21.987836 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20289c-04bf-420b-ac4d-68a88fee0e18 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:21.989426 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 927/1834] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:21 2022] POST /identity/v3/projects => generated 381 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:21.994748 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6225d57-bfba-4c53-871e-d0e36927a8ac admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:22.036434 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 908/1835] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:21 2022] PUT /identity/v3/projects/d5d6cbe484a7475ea280b15b7a798789/users/89b164163ed842c1869a2233212a3aa2/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:22.038215 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55088e34-3062-4b33-ad64-1279de79440b None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:22.038679 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55088e34-3062-4b33-ad64-1279de79440b None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:22.038956 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55088e34-3062-4b33-ad64-1279de79440b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:22.114828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-55088e34-3062-4b33-ad64-1279de79440b None None] MFA Rules not processed for user `89b164163ed842c1869a2233212a3aa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:22.116169 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-55088e34-3062-4b33-ad64-1279de79440b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:22.164570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6225d57-bfba-4c53-871e-d0e36927a8ac admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:22.284241 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 909/1836] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:22.293751 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55088e34-3062-4b33-ad64-1279de79440b None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:22.372694 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77747a93-fb6f-4b4a-818f-6026fe1b19c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:22.373660 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77747a93-fb6f-4b4a-818f-6026fe1b19c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:22.379139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77747a93-fb6f-4b4a-818f-6026fe1b19c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:22.379523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77747a93-fb6f-4b4a-818f-6026fe1b19c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:22.380710 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77747a93-fb6f-4b4a-818f-6026fe1b19c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:22.384611 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77747a93-fb6f-4b4a-818f-6026fe1b19c4 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:22.410936 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:22.439901 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 928/1837] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:21 2022] POST /identity/v3/users => generated 297 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:22.445415 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77747a93-fb6f-4b4a-818f-6026fe1b19c4 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:22.484842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77747a93-fb6f-4b4a-818f-6026fe1b19c4 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:22.491155 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55088e34-3062-4b33-ad64-1279de79440b None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:22.496428 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6945a-dde5-4920-b82a-f6d32cf70941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:22.497038 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f318a-6941-4f31-b3b6-a0d7e87e6bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:22.497424 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6945a-dde5-4920-b82a-f6d32cf70941 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:22.497772 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f318a-6941-4f31-b3b6-a0d7e87e6bbb admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:22.498153 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6945a-dde5-4920-b82a-f6d32cf70941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:22.498272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f318a-6941-4f31-b3b6-a0d7e87e6bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:22.498676 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6945a-dde5-4920-b82a-f6d32cf70941 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:22.499596 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6945a-dde5-4920-b82a-f6d32cf70941 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:22.501717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6945a-dde5-4920-b82a-f6d32cf70941 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:22.504183 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f318a-6941-4f31-b3b6-a0d7e87e6bbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:22.505144 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38f318a-6941-4f31-b3b6-a0d7e87e6bbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:22.507232 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38f318a-6941-4f31-b3b6-a0d7e87e6bbb admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:22.517831 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 929/1838] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:22.519130 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6945a-dde5-4920-b82a-f6d32cf70941 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:22.521816 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-96f6945a-dde5-4920-b82a-f6d32cf70941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:22.548892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6945a-dde5-4920-b82a-f6d32cf70941 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:22.554877 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7407debc-9af8-44db-85f7-778135b817f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:22.556082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407debc-9af8-44db-85f7-778135b817f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:22.556730 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407debc-9af8-44db-85f7-778135b817f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:22.557181 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407debc-9af8-44db-85f7-778135b817f2 admin admin] PATH_INFO: `/v3/users/a13a0b07b3394507b7dc4849f0f0eaa1` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:22.557293 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 910/1839] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:22 2022] POST /identity/v3/projects => generated 365 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:22.558319 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f318a-6941-4f31-b3b6-a0d7e87e6bbb admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:22.560146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7407debc-9af8-44db-85f7-778135b817f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a13a0b07b3394507b7dc4849f0f0eaa1)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:22.562850 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7407debc-9af8-44db-85f7-778135b817f2 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:22.628367 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:22.653865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-7407debc-9af8-44db-85f7-778135b817f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a13a0b07b3394507b7dc4849f0f0eaa1'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:22.670805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-7407debc-9af8-44db-85f7-778135b817f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a13a0b07b3394507b7dc4849f0f0eaa1'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:22.672764 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-7407debc-9af8-44db-85f7-778135b817f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a13a0b07b3394507b7dc4849f0f0eaa1'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:22.690758 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f318a-6941-4f31-b3b6-a0d7e87e6bbb admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:22.697374 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a69679-18e6-42c7-93bb-70bee918713b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:22.698398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a69679-18e6-42c7-93bb-70bee918713b admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:22.698787 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a69679-18e6-42c7-93bb-70bee918713b admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:22.699331 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a69679-18e6-42c7-93bb-70bee918713b admin admin] PATH_INFO: `/v3/projects/174aedf374c54b5fa33d033e3f34df24/users/5f8f9648acec4f7a930f2e662143468e/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:22.702872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-7407debc-9af8-44db-85f7-778135b817f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a13a0b07b3394507b7dc4849f0f0eaa1'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:22.706798 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-7407debc-9af8-44db-85f7-778135b817f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a13a0b07b3394507b7dc4849f0f0eaa1'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:22.722971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a69679-18e6-42c7-93bb-70bee918713b admin admin] RBAC: Authorizing `identity:create_grant(project_id=174aedf374c54b5fa33d033e3f34df24, user_id=5f8f9648acec4f7a930f2e662143468e, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:22.729652 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a69679-18e6-42c7-93bb-70bee918713b admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:22.739004 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 930/1840] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:22 2022] DELETE /identity/v3/users/a13a0b07b3394507b7dc4849f0f0eaa1 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:22.743233 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7407debc-9af8-44db-85f7-778135b817f2 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:22.748764 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7407debc-9af8-44db-85f7-778135b817f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:22.750174 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 911/1841] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:22 2022] PUT /identity/v3/projects/174aedf374c54b5fa33d033e3f34df24/users/5f8f9648acec4f7a930f2e662143468e/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:22.751835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014663f7-d4d3-43aa-9188-9d3fe6c103f2 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:22.752223 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014663f7-d4d3-43aa-9188-9d3fe6c103f2 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:22.752596 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014663f7-d4d3-43aa-9188-9d3fe6c103f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:22.784481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-014663f7-d4d3-43aa-9188-9d3fe6c103f2 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:22.786481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-014663f7-d4d3-43aa-9188-9d3fe6c103f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:22.903504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 912/1842] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:22.903504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014663f7-d4d3-43aa-9188-9d3fe6c103f2 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:22.907087 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7407debc-9af8-44db-85f7-778135b817f2 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:22.908031 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-014663f7-d4d3-43aa-9188-9d3fe6c103f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:22.915389 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518e8d9-7fcb-4f92-bab9-4600f2b18dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:22.916031 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518e8d9-7fcb-4f92-bab9-4600f2b18dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:22.916635 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518e8d9-7fcb-4f92-bab9-4600f2b18dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:22.916913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518e8d9-7fcb-4f92-bab9-4600f2b18dd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:22.917689 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518e8d9-7fcb-4f92-bab9-4600f2b18dd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:22.921344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518e8d9-7fcb-4f92-bab9-4600f2b18dd8 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:22.944770 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014663f7-d4d3-43aa-9188-9d3fe6c103f2 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:22.960630 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83aadd8-0aef-4c83-9f58-770ea604dc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:22.961971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83aadd8-0aef-4c83-9f58-770ea604dc6c admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:22.962261 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83aadd8-0aef-4c83-9f58-770ea604dc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:22.962535 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83aadd8-0aef-4c83-9f58-770ea604dc6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:22.963248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83aadd8-0aef-4c83-9f58-770ea604dc6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:22.964233 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 931/1843] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:22 2022] POST /identity/v3/users => generated 289 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:22.965467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518e8d9-7fcb-4f92-bab9-4600f2b18dd8 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:22.967563 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83aadd8-0aef-4c83-9f58-770ea604dc6c admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:22.969293 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2518e8d9-7fcb-4f92-bab9-4600f2b18dd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:23.001867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 913/1844] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:23.003100 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83aadd8-0aef-4c83-9f58-770ea604dc6c admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:23.013733 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83aadd8-0aef-4c83-9f58-770ea604dc6c admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:23.023288 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c6722-1914-4cfc-bb28-c5a4f7486c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:23.024187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c6722-1914-4cfc-bb28-c5a4f7486c35 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:23.024615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c6722-1914-4cfc-bb28-c5a4f7486c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:23.025121 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c6722-1914-4cfc-bb28-c5a4f7486c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:23.026125 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c6722-1914-4cfc-bb28-c5a4f7486c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:23.031150 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c6722-1914-4cfc-bb28-c5a4f7486c35 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:23.053806 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 914/1845] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:23.055432 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c6722-1914-4cfc-bb28-c5a4f7486c35 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:23.058461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518e8d9-7fcb-4f92-bab9-4600f2b18dd8 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:23.069699 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b79f2-5c7a-4fa2-9d18-0c2ec3e7f415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:23.071866 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b79f2-5c7a-4fa2-9d18-0c2ec3e7f415 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:23.072274 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b79f2-5c7a-4fa2-9d18-0c2ec3e7f415 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:23.073580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b79f2-5c7a-4fa2-9d18-0c2ec3e7f415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:23.076046 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c6722-1914-4cfc-bb28-c5a4f7486c35 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:23.083119 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df9341-50fd-4935-b4c4-6a3aac42419d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:23.084534 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df9341-50fd-4935-b4c4-6a3aac42419d admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:23.084899 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df9341-50fd-4935-b4c4-6a3aac42419d admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:23.085298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df9341-50fd-4935-b4c4-6a3aac42419d admin admin] PATH_INFO: `/v3/projects/174aedf374c54b5fa33d033e3f34df24/users/5f8f9648acec4f7a930f2e662143468e/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:23.088547 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df9341-50fd-4935-b4c4-6a3aac42419d admin admin] RBAC: Authorizing `identity:create_grant(project_id=174aedf374c54b5fa33d033e3f34df24, user_id=5f8f9648acec4f7a930f2e662143468e, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:23.093711 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df9341-50fd-4935-b4c4-6a3aac42419d admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:23.107579 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b79f2-5c7a-4fa2-9d18-0c2ec3e7f415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:23.112984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b79f2-5c7a-4fa2-9d18-0c2ec3e7f415 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:23.128076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 915/1846] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:23 2022] PUT /identity/v3/projects/174aedf374c54b5fa33d033e3f34df24/users/5f8f9648acec4f7a930f2e662143468e/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:23.138915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df9341-50fd-4935-b4c4-6a3aac42419d admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:23.145663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 932/1847] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:23.148131 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc5f12-fa3e-4cb5-973a-d37563fd3820 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:23.148529 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc5f12-fa3e-4cb5-973a-d37563fd3820 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:23.148949 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc5f12-fa3e-4cb5-973a-d37563fd3820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:23.161372 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:23.237484 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-87cc5f12-fa3e-4cb5-973a-d37563fd3820 None None] MFA Rules not processed for user `5f8f9648acec4f7a930f2e662143468e`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:23.244017 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-87cc5f12-fa3e-4cb5-973a-d37563fd3820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:23.292407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df9341-50fd-4935-b4c4-6a3aac42419d admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:23.303099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2848248-ee1b-45c1-ab15-efdfb3fa99b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:23.305355 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2848248-ee1b-45c1-ab15-efdfb3fa99b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:23.306315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2848248-ee1b-45c1-ab15-efdfb3fa99b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:23.307847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2848248-ee1b-45c1-ab15-efdfb3fa99b5 admin admin] PATH_INFO: `/v3/projects/f422b8db0ca14359b538c1d6787ff303/users/62d464ca34534d98858e7e4138a67225/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:23.334349 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2848248-ee1b-45c1-ab15-efdfb3fa99b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f422b8db0ca14359b538c1d6787ff303, user_id=62d464ca34534d98858e7e4138a67225, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:23.337685 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2848248-ee1b-45c1-ab15-efdfb3fa99b5 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:23.367310 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 916/1848] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:23 2022] PUT /identity/v3/projects/f422b8db0ca14359b538c1d6787ff303/users/62d464ca34534d98858e7e4138a67225/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:23.372488 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:23.378815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2848248-ee1b-45c1-ab15-efdfb3fa99b5 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:23.481979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 933/1849] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:23.498983 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc5f12-fa3e-4cb5-973a-d37563fd3820 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:23.507184 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-87cc5f12-fa3e-4cb5-973a-d37563fd3820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:23.523622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2848248-ee1b-45c1-ab15-efdfb3fa99b5 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:23.533016 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c1da14-67c7-497f-8015-914a82ffb723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:23.534060 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1da14-67c7-497f-8015-914a82ffb723 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:23.534608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1da14-67c7-497f-8015-914a82ffb723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:23.535155 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1da14-67c7-497f-8015-914a82ffb723 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:23.535981 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c1da14-67c7-497f-8015-914a82ffb723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:23.537723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c1da14-67c7-497f-8015-914a82ffb723 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:23.555061 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 917/1850] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:23.558242 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c1da14-67c7-497f-8015-914a82ffb723 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:23.570958 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c1da14-67c7-497f-8015-914a82ffb723 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:23.579586 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5970da-3115-4cdf-bb23-d81eba2c987f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:23.580375 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5970da-3115-4cdf-bb23-d81eba2c987f admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:23.580656 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5970da-3115-4cdf-bb23-d81eba2c987f admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:23.580930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5970da-3115-4cdf-bb23-d81eba2c987f admin admin] PATH_INFO: `/v3/projects/f422b8db0ca14359b538c1d6787ff303/users/62d464ca34534d98858e7e4138a67225/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:23.583454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5970da-3115-4cdf-bb23-d81eba2c987f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f422b8db0ca14359b538c1d6787ff303, user_id=62d464ca34534d98858e7e4138a67225, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:23.586834 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5970da-3115-4cdf-bb23-d81eba2c987f admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:23.600171 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 918/1851] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:23 2022] PUT /identity/v3/projects/f422b8db0ca14359b538c1d6787ff303/users/62d464ca34534d98858e7e4138a67225/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:23.612421 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc5f12-fa3e-4cb5-973a-d37563fd3820 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:23.612825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf317c3-b49c-4464-84d6-afc3f7dd2e6b None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:23.613110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf317c3-b49c-4464-84d6-afc3f7dd2e6b None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:23.613381 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf317c3-b49c-4464-84d6-afc3f7dd2e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:23.654153 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-edf317c3-b49c-4464-84d6-afc3f7dd2e6b None None] MFA Rules not processed for user `62d464ca34534d98858e7e4138a67225`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:23.655385 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-edf317c3-b49c-4464-84d6-afc3f7dd2e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:23.697198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e73435-690a-4c93-bf3e-663819802963 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:23.697878 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e73435-690a-4c93-bf3e-663819802963 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:23.698286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e73435-690a-4c93-bf3e-663819802963 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:23.698582 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e73435-690a-4c93-bf3e-663819802963 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:23.760139 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:23.765429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 919/1852] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:23 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:39:23.779247 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf317c3-b49c-4464-84d6-afc3f7dd2e6b None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:23.780760 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-edf317c3-b49c-4464-84d6-afc3f7dd2e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:23.794587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf317c3-b49c-4464-84d6-afc3f7dd2e6b None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:23.797779 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e73435-690a-4c93-bf3e-663819802963 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:23.799408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e73435-690a-4c93-bf3e-663819802963 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:23.799626 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63054b71-6e76-4428-aace-6a4882e11ee2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:23.800235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63054b71-6e76-4428-aace-6a4882e11ee2 service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:23.800516 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63054b71-6e76-4428-aace-6a4882e11ee2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:23.800803 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63054b71-6e76-4428-aace-6a4882e11ee2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:23.812875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63054b71-6e76-4428-aace-6a4882e11ee2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:23.813283 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 934/1853] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:23.814405 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63054b71-6e76-4428-aace-6a4882e11ee2 service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:23.828944 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 920/1854] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:23 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:39:23.925608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e73435-690a-4c93-bf3e-663819802963 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:23.933224 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e73435-690a-4c93-bf3e-663819802963 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:23.938134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7e289-3eaf-478c-80a2-62a138945bc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:23.938767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7e289-3eaf-478c-80a2-62a138945bc9 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:23.939049 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7e289-3eaf-478c-80a2-62a138945bc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:23.939632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7e289-3eaf-478c-80a2-62a138945bc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:23.941633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-abc7e289-3eaf-478c-80a2-62a138945bc9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:23.995263 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:24.050323 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7e289-3eaf-478c-80a2-62a138945bc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:24.052428 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7e289-3eaf-478c-80a2-62a138945bc9 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:24.079208 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 935/1855] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:24.117361 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63054b71-6e76-4428-aace-6a4882e11ee2 service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:24.118970 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-63054b71-6e76-4428-aace-6a4882e11ee2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:24.230530 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63054b71-6e76-4428-aace-6a4882e11ee2 service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:24.256800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536fd0f-963a-48c1-8f8c-ff241bd34eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:24.257491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536fd0f-963a-48c1-8f8c-ff241bd34eca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:24.257800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536fd0f-963a-48c1-8f8c-ff241bd34eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:24.258075 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536fd0f-963a-48c1-8f8c-ff241bd34eca admin admin] PATH_INFO: `/v3/projects/6fa49c2084db42859cab0a2fbe5454db` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:24.259316 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f536fd0f-963a-48c1-8f8c-ff241bd34eca admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fa49c2084db42859cab0a2fbe5454db)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:24.261364 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f536fd0f-963a-48c1-8f8c-ff241bd34eca admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:24.335134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7e289-3eaf-478c-80a2-62a138945bc9 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:24.350184 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7e289-3eaf-478c-80a2-62a138945bc9 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:24.357795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-f536fd0f-963a-48c1-8f8c-ff241bd34eca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fa49c2084db42859cab0a2fbe5454db', 'request_id': 'req-f536fd0f-963a-48c1-8f8c-ff241bd34eca'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:24.372531 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-f536fd0f-963a-48c1-8f8c-ff241bd34eca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fa49c2084db42859cab0a2fbe5454db', 'request_id': 'req-f536fd0f-963a-48c1-8f8c-ff241bd34eca'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:24.391065 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-f536fd0f-963a-48c1-8f8c-ff241bd34eca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fa49c2084db42859cab0a2fbe5454db', 'request_id': 'req-f536fd0f-963a-48c1-8f8c-ff241bd34eca'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:24.408656 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-f536fd0f-963a-48c1-8f8c-ff241bd34eca admin admin] The token cache is being invalidate because project 6fa49c2084db42859cab0a2fbe5454db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:24.409140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-f536fd0f-963a-48c1-8f8c-ff241bd34eca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:24.410985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-f536fd0f-963a-48c1-8f8c-ff241bd34eca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:24.413924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 921/1856] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:24 2022] DELETE /identity/v3/projects/6fa49c2084db42859cab0a2fbe5454db => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:24.418324 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536fd0f-963a-48c1-8f8c-ff241bd34eca admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:24.420965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-f536fd0f-963a-48c1-8f8c-ff241bd34eca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:24.449150 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36af83e-4ec5-42c3-af59-1900ce62a7b8 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8f9648acec4f7a930f2e662143468e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '174aedf374c54b5fa33d033e3f34df24', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:24.449888 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36af83e-4ec5-42c3-af59-1900ce62a7b8 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:24.450174 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36af83e-4ec5-42c3-af59-1900ce62a7b8 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:24.450460 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36af83e-4ec5-42c3-af59-1900ce62a7b8 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:24.452105 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36af83e-4ec5-42c3-af59-1900ce62a7b8 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:39:24.452425 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36af83e-4ec5-42c3-af59-1900ce62a7b8 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:24.454163 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36af83e-4ec5-42c3-af59-1900ce62a7b8 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:24.473942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 936/1857] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:39:24.478577 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36af83e-4ec5-42c3-af59-1900ce62a7b8 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:24.484608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c36af83e-4ec5-42c3-af59-1900ce62a7b8 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:24.504763 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36af83e-4ec5-42c3-af59-1900ce62a7b8 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:24.514143 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d88d881-6882-4d41-8294-ec5991376f72 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8f9648acec4f7a930f2e662143468e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '174aedf374c54b5fa33d033e3f34df24', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:24.514956 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88d881-6882-4d41-8294-ec5991376f72 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:24.515311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88d881-6882-4d41-8294-ec5991376f72 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:24.515661 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88d881-6882-4d41-8294-ec5991376f72 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:24.516405 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d88d881-6882-4d41-8294-ec5991376f72 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:24.518869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d88d881-6882-4d41-8294-ec5991376f72 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:24.526515 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536fd0f-963a-48c1-8f8c-ff241bd34eca admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:24.549911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f3ee1-1237-4d92-88df-061486d4abb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:24.551034 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f3ee1-1237-4d92-88df-061486d4abb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:24.551473 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f3ee1-1237-4d92-88df-061486d4abb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:24.551909 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f3ee1-1237-4d92-88df-061486d4abb0 admin admin] PATH_INFO: `/v3/users/b3de9bce01c94017b921b6ddba404854` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:24.555530 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f3ee1-1237-4d92-88df-061486d4abb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3de9bce01c94017b921b6ddba404854)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:24.559135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f3ee1-1237-4d92-88df-061486d4abb0 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:24.574283 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 937/1858] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:24 2022] POST /identity/v3/projects => generated 353 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:24.575640 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d88d881-6882-4d41-8294-ec5991376f72 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:24.594607 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:24.633721 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9c6f3ee1-1237-4d92-88df-061486d4abb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3de9bce01c94017b921b6ddba404854'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:24.638177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9c6f3ee1-1237-4d92-88df-061486d4abb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3de9bce01c94017b921b6ddba404854'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:24.656785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9c6f3ee1-1237-4d92-88df-061486d4abb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3de9bce01c94017b921b6ddba404854'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:24.666153 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9c6f3ee1-1237-4d92-88df-061486d4abb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3de9bce01c94017b921b6ddba404854'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:24.667088 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9c6f3ee1-1237-4d92-88df-061486d4abb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3de9bce01c94017b921b6ddba404854'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:24.683977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 922/1859] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:24 2022] DELETE /identity/v3/users/b3de9bce01c94017b921b6ddba404854 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:24.684598 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f3ee1-1237-4d92-88df-061486d4abb0 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:24.686770 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-9c6f3ee1-1237-4d92-88df-061486d4abb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:24.713321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d88d881-6882-4d41-8294-ec5991376f72 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:24.798920 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f3ee1-1237-4d92-88df-061486d4abb0 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:24.811755 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816530b-fdb9-401d-8a31-cde2ffdc6bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:24.812433 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816530b-fdb9-401d-8a31-cde2ffdc6bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:24.812713 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816530b-fdb9-401d-8a31-cde2ffdc6bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:24.812987 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816530b-fdb9-401d-8a31-cde2ffdc6bcc admin admin] PATH_INFO: `/v3/users/d8c974564c6f4c168cca5cec77426e98` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:24.816431 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816530b-fdb9-401d-8a31-cde2ffdc6bcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8c974564c6f4c168cca5cec77426e98)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:24.819638 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816530b-fdb9-401d-8a31-cde2ffdc6bcc admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:24.866303 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:24.878623 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33624e38-c9d1-45aa-b18a-a3d1dd6eb7e7 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8f9648acec4f7a930f2e662143468e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '174aedf374c54b5fa33d033e3f34df24', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:24.880092 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33624e38-c9d1-45aa-b18a-a3d1dd6eb7e7 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:24.880582 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33624e38-c9d1-45aa-b18a-a3d1dd6eb7e7 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:24.880841 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33624e38-c9d1-45aa-b18a-a3d1dd6eb7e7 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:24.881753 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33624e38-c9d1-45aa-b18a-a3d1dd6eb7e7 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:39:24.882192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33624e38-c9d1-45aa-b18a-a3d1dd6eb7e7 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:24.884245 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33624e38-c9d1-45aa-b18a-a3d1dd6eb7e7 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:24.930626 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-1816530b-fdb9-401d-8a31-cde2ffdc6bcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8c974564c6f4c168cca5cec77426e98'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:24.932785 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 938/1860] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:39:24.935912 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-1816530b-fdb9-401d-8a31-cde2ffdc6bcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8c974564c6f4c168cca5cec77426e98'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:24.961615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-1816530b-fdb9-401d-8a31-cde2ffdc6bcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8c974564c6f4c168cca5cec77426e98'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:24.974051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-1816530b-fdb9-401d-8a31-cde2ffdc6bcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8c974564c6f4c168cca5cec77426e98'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:24.976314 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-1816530b-fdb9-401d-8a31-cde2ffdc6bcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8c974564c6f4c168cca5cec77426e98'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:24.995869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33624e38-c9d1-45aa-b18a-a3d1dd6eb7e7 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:24.998534 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-33624e38-c9d1-45aa-b18a-a3d1dd6eb7e7 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:25.004927 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 923/1861] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:24 2022] DELETE /identity/v3/users/d8c974564c6f4c168cca5cec77426e98 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:25.009600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3f57f-cf08-4c72-a13b-b2b175025463 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:25.009974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3f57f-cf08-4c72-a13b-b2b175025463 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:25.010318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3f57f-cf08-4c72-a13b-b2b175025463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:25.015843 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33624e38-c9d1-45aa-b18a-a3d1dd6eb7e7 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:25.033187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-74c3f57f-cf08-4c72-a13b-b2b175025463 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:25.034876 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-74c3f57f-cf08-4c72-a13b-b2b175025463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:25.048983 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:25.101329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354f29a-c8cb-44ba-949b-70835dfcbef2 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8f9648acec4f7a930f2e662143468e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '174aedf374c54b5fa33d033e3f34df24', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:25.102337 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354f29a-c8cb-44ba-949b-70835dfcbef2 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:25.102850 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354f29a-c8cb-44ba-949b-70835dfcbef2 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:25.104287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354f29a-c8cb-44ba-949b-70835dfcbef2 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] PATH_INFO: `/v3/projects/7934c84fa2434125885b997546082a5c` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:25.105358 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354f29a-c8cb-44ba-949b-70835dfcbef2 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7934c84fa2434125885b997546082a5c)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:25.108882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354f29a-c8cb-44ba-949b-70835dfcbef2 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:25.113271 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 924/1862] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:39:25.147010 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3f57f-cf08-4c72-a13b-b2b175025463 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:25.148654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-74c3f57f-cf08-4c72-a13b-b2b175025463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:25.194348 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-3354f29a-c8cb-44ba-949b-70835dfcbef2 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7934c84fa2434125885b997546082a5c', 'request_id': 'req-3354f29a-c8cb-44ba-949b-70835dfcbef2'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:25.208001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-3354f29a-c8cb-44ba-949b-70835dfcbef2 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7934c84fa2434125885b997546082a5c', 'request_id': 'req-3354f29a-c8cb-44ba-949b-70835dfcbef2'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:25.225568 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-3354f29a-c8cb-44ba-949b-70835dfcbef2 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7934c84fa2434125885b997546082a5c', 'request_id': 'req-3354f29a-c8cb-44ba-949b-70835dfcbef2'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:25.264401 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-3354f29a-c8cb-44ba-949b-70835dfcbef2 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] The token cache is being invalidate because project 7934c84fa2434125885b997546082a5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:25.264401 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-3354f29a-c8cb-44ba-949b-70835dfcbef2 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:25.265029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3f57f-cf08-4c72-a13b-b2b175025463 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:25.265164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-3354f29a-c8cb-44ba-949b-70835dfcbef2 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:25.265164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 939/1863] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:24 2022] DELETE /identity/v3/projects/7934c84fa2434125885b997546082a5c => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:39:25.356361 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16daedb9-3175-40ce-8747-c477c6741285 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:25.357697 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16daedb9-3175-40ce-8747-c477c6741285 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:25.357870 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16daedb9-3175-40ce-8747-c477c6741285 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:25.358178 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16daedb9-3175-40ce-8747-c477c6741285 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:25.360836 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-16daedb9-3175-40ce-8747-c477c6741285 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:25.534051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16daedb9-3175-40ce-8747-c477c6741285 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:25.535593 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16daedb9-3175-40ce-8747-c477c6741285 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:25.564238 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 925/1864] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:39:25.658736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354f29a-c8cb-44ba-949b-70835dfcbef2 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:25.660235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3354f29a-c8cb-44ba-949b-70835dfcbef2 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:25.679270 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354f29a-c8cb-44ba-949b-70835dfcbef2 tempest-VolumeQuotaClassesTest-1082440259 tempest-VolumeQuotaClassesTest-1082440259-project-admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:25.687466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb950a-22fc-4555-b9c7-f35210bd00bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:25.688187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb950a-22fc-4555-b9c7-f35210bd00bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:25.688667 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb950a-22fc-4555-b9c7-f35210bd00bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:25.688955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb950a-22fc-4555-b9c7-f35210bd00bf admin admin] PATH_INFO: `/v3/users/5acfefd72ff947f1b40a926739419409` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:25.691127 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb950a-22fc-4555-b9c7-f35210bd00bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5acfefd72ff947f1b40a926739419409)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:25.694890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb950a-22fc-4555-b9c7-f35210bd00bf admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:25.867712 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-debb950a-22fc-4555-b9c7-f35210bd00bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5acfefd72ff947f1b40a926739419409'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:25.894364 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-debb950a-22fc-4555-b9c7-f35210bd00bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5acfefd72ff947f1b40a926739419409'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:25.896074 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-debb950a-22fc-4555-b9c7-f35210bd00bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5acfefd72ff947f1b40a926739419409'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:25.922916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-debb950a-22fc-4555-b9c7-f35210bd00bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5acfefd72ff947f1b40a926739419409'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:25.925259 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-debb950a-22fc-4555-b9c7-f35210bd00bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5acfefd72ff947f1b40a926739419409'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:25.930690 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:25.959626 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 940/1865] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:25 2022] DELETE /identity/v3/users/5acfefd72ff947f1b40a926739419409 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:25.964601 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6c058-7c39-4423-80bd-1c1f1c1bc4bb None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:25.965585 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6c058-7c39-4423-80bd-1c1f1c1bc4bb None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:25.965585 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6c058-7c39-4423-80bd-1c1f1c1bc4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:26.017581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-55a6c058-7c39-4423-80bd-1c1f1c1bc4bb None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:26.023020 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-55a6c058-7c39-4423-80bd-1c1f1c1bc4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:26.063141 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb950a-22fc-4555-b9c7-f35210bd00bf admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:26.064674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-debb950a-22fc-4555-b9c7-f35210bd00bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:26.090629 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:26.277832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 926/1866] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:26.289890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb950a-22fc-4555-b9c7-f35210bd00bf admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:26.298413 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a6c058-7c39-4423-80bd-1c1f1c1bc4bb None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:26.300440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-55a6c058-7c39-4423-80bd-1c1f1c1bc4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:26.310207 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9794981-7008-44fd-97b7-4f6e56c3e535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:26.311006 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9794981-7008-44fd-97b7-4f6e56c3e535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:26.311371 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9794981-7008-44fd-97b7-4f6e56c3e535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:26.311725 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9794981-7008-44fd-97b7-4f6e56c3e535 admin admin] PATH_INFO: `/v3/projects/8a9a5b74819b4bda926c9f5345ee14d0` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:26.316666 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9794981-7008-44fd-97b7-4f6e56c3e535 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a9a5b74819b4bda926c9f5345ee14d0)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:26.323235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9794981-7008-44fd-97b7-4f6e56c3e535 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:26.426335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e9794981-7008-44fd-97b7-4f6e56c3e535 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a9a5b74819b4bda926c9f5345ee14d0', 'request_id': 'req-e9794981-7008-44fd-97b7-4f6e56c3e535'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:26.446825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a6c058-7c39-4423-80bd-1c1f1c1bc4bb None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:26.447521 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e9794981-7008-44fd-97b7-4f6e56c3e535 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a9a5b74819b4bda926c9f5345ee14d0', 'request_id': 'req-e9794981-7008-44fd-97b7-4f6e56c3e535'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:26.455446 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e9794981-7008-44fd-97b7-4f6e56c3e535 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a9a5b74819b4bda926c9f5345ee14d0', 'request_id': 'req-e9794981-7008-44fd-97b7-4f6e56c3e535'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:26.502499 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e9794981-7008-44fd-97b7-4f6e56c3e535 admin admin] The token cache is being invalidate because project 8a9a5b74819b4bda926c9f5345ee14d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:26.502719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e9794981-7008-44fd-97b7-4f6e56c3e535 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:26.503670 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e9794981-7008-44fd-97b7-4f6e56c3e535 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:26.508571 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 941/1867] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:26 2022] DELETE /identity/v3/projects/8a9a5b74819b4bda926c9f5345ee14d0 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:39:26.515697 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74625d-6a93-4106-a1f8-47cc740e04a5 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:26.515986 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74625d-6a93-4106-a1f8-47cc740e04a5 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:26.516261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74625d-6a93-4106-a1f8-47cc740e04a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:26.574973 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085b647-3dda-4330-a863-efbdd948a603 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:26.575547 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085b647-3dda-4330-a863-efbdd948a603 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:26.575547 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085b647-3dda-4330-a863-efbdd948a603 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:26.575935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085b647-3dda-4330-a863-efbdd948a603 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:26.581410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c085b647-3dda-4330-a863-efbdd948a603 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:26.581708 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-3f74625d-6a93-4106-a1f8-47cc740e04a5 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:26.582918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3f74625d-6a93-4106-a1f8-47cc740e04a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:26.727925 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085b647-3dda-4330-a863-efbdd948a603 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:26.729506 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085b647-3dda-4330-a863-efbdd948a603 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:26.730342 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 942/1868] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:39:26.731383 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74625d-6a93-4106-a1f8-47cc740e04a5 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:26.733496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3f74625d-6a93-4106-a1f8-47cc740e04a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:26.744617 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 927/1869] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:39:26.746337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae05b0b-88cc-4bda-938f-0fba46b18e05 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:26.746627 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae05b0b-88cc-4bda-938f-0fba46b18e05 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:26.748607 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae05b0b-88cc-4bda-938f-0fba46b18e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:26.759777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74625d-6a93-4106-a1f8-47cc740e04a5 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:26.768290 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941f67b-0d47-417a-b79b-b2e844984ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:26.769231 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941f67b-0d47-417a-b79b-b2e844984ed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:26.769751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941f67b-0d47-417a-b79b-b2e844984ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:26.771890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941f67b-0d47-417a-b79b-b2e844984ed9 admin admin] PATH_INFO: `/v3/projects/695505cc14124b4bb4378ba771624352` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:26.773854 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8941f67b-0d47-417a-b79b-b2e844984ed9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=695505cc14124b4bb4378ba771624352)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:26.777658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8941f67b-0d47-417a-b79b-b2e844984ed9 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:26.780342 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-4ae05b0b-88cc-4bda-938f-0fba46b18e05 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:26.782110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-4ae05b0b-88cc-4bda-938f-0fba46b18e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:26.796856 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 928/1870] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:26.798976 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae05b0b-88cc-4bda-938f-0fba46b18e05 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:26.810002 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae05b0b-88cc-4bda-938f-0fba46b18e05 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:26.814761 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:26.816533 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1bbb0-7814-44ad-8152-56d6f456eb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:26.817650 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1bbb0-7814-44ad-8152-56d6f456eb24 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:26.818236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1bbb0-7814-44ad-8152-56d6f456eb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:26.818517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1bbb0-7814-44ad-8152-56d6f456eb24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:26.819484 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a1bbb0-7814-44ad-8152-56d6f456eb24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:39:26.819704 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a1bbb0-7814-44ad-8152-56d6f456eb24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:26.821341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a1bbb0-7814-44ad-8152-56d6f456eb24 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:26.845168 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 929/1871] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:39:26.849940 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1bbb0-7814-44ad-8152-56d6f456eb24 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:26.867878 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1bbb0-7814-44ad-8152-56d6f456eb24 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:26.882204 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8941f67b-0d47-417a-b79b-b2e844984ed9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '695505cc14124b4bb4378ba771624352', 'request_id': 'req-8941f67b-0d47-417a-b79b-b2e844984ed9'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:26.885125 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:26.897358 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8941f67b-0d47-417a-b79b-b2e844984ed9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '695505cc14124b4bb4378ba771624352', 'request_id': 'req-8941f67b-0d47-417a-b79b-b2e844984ed9'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:26.911610 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8941f67b-0d47-417a-b79b-b2e844984ed9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '695505cc14124b4bb4378ba771624352', 'request_id': 'req-8941f67b-0d47-417a-b79b-b2e844984ed9'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:26.943171 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8941f67b-0d47-417a-b79b-b2e844984ed9 admin admin] The token cache is being invalidate because project 695505cc14124b4bb4378ba771624352 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:26.943559 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8941f67b-0d47-417a-b79b-b2e844984ed9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:26.945169 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8941f67b-0d47-417a-b79b-b2e844984ed9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:26.949772 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 943/1872] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:26 2022] DELETE /identity/v3/projects/695505cc14124b4bb4378ba771624352 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:26.951243 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941f67b-0d47-417a-b79b-b2e844984ed9 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:26.953164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8941f67b-0d47-417a-b79b-b2e844984ed9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:27.026289 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be642078-85a8-4648-9b44-220afe2759b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:27.027251 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be642078-85a8-4648-9b44-220afe2759b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:27.027925 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be642078-85a8-4648-9b44-220afe2759b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:27.028308 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be642078-85a8-4648-9b44-220afe2759b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:27.030489 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be642078-85a8-4648-9b44-220afe2759b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:27.036437 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be642078-85a8-4648-9b44-220afe2759b2 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:27.105394 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 930/1873] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:26 2022] POST /identity/v3/projects => generated 379 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 04:39:27.110324 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be642078-85a8-4648-9b44-220afe2759b2 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:27.112791 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-be642078-85a8-4648-9b44-220afe2759b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:27.123364 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941f67b-0d47-417a-b79b-b2e844984ed9 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:27.280437 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0524b14-02cf-436e-ac3f-09695bb27fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:27.282120 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0524b14-02cf-436e-ac3f-09695bb27fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:27.282942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0524b14-02cf-436e-ac3f-09695bb27fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:27.282942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0524b14-02cf-436e-ac3f-09695bb27fc7 admin admin] PATH_INFO: `/v3/users/59c0b3df618d40948b8ab1f033cf5089` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:27.285016 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0524b14-02cf-436e-ac3f-09695bb27fc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59c0b3df618d40948b8ab1f033cf5089)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:27.287029 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0524b14-02cf-436e-ac3f-09695bb27fc7 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:27.360109 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b0524b14-02cf-436e-ac3f-09695bb27fc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59c0b3df618d40948b8ab1f033cf5089'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:27.368547 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:27.370899 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b0524b14-02cf-436e-ac3f-09695bb27fc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59c0b3df618d40948b8ab1f033cf5089'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:27.371830 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b0524b14-02cf-436e-ac3f-09695bb27fc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59c0b3df618d40948b8ab1f033cf5089'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:27.388878 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b0524b14-02cf-436e-ac3f-09695bb27fc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59c0b3df618d40948b8ab1f033cf5089'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:27.390362 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b0524b14-02cf-436e-ac3f-09695bb27fc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59c0b3df618d40948b8ab1f033cf5089'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:27.407730 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be642078-85a8-4648-9b44-220afe2759b2 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:27.417098 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 944/1874] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:26 2022] DELETE /identity/v3/users/59c0b3df618d40948b8ab1f033cf5089 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:39:27.421378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a628cdc-46cd-45f7-b82f-d5c00e94984f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:27.422352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a628cdc-46cd-45f7-b82f-d5c00e94984f admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:27.423208 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a628cdc-46cd-45f7-b82f-d5c00e94984f admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:27.423739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a628cdc-46cd-45f7-b82f-d5c00e94984f admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:27.424641 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a628cdc-46cd-45f7-b82f-d5c00e94984f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:27.427223 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a628cdc-46cd-45f7-b82f-d5c00e94984f admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:27.459607 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 931/1875] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:27 2022] POST /identity/v3/users => generated 290 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:27.463940 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0524b14-02cf-436e-ac3f-09695bb27fc7 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:27.467081 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b0524b14-02cf-436e-ac3f-09695bb27fc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:27.560378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a628cdc-46cd-45f7-b82f-d5c00e94984f admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:27.564165 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-1a628cdc-46cd-45f7-b82f-d5c00e94984f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:27.579808 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a628cdc-46cd-45f7-b82f-d5c00e94984f admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:27.620150 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0524b14-02cf-436e-ac3f-09695bb27fc7 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:27.625819 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a26c5-030d-4f8a-a500-1122c89d34ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:27.626473 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a26c5-030d-4f8a-a500-1122c89d34ce admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:27.627878 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a26c5-030d-4f8a-a500-1122c89d34ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:27.628204 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a26c5-030d-4f8a-a500-1122c89d34ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:27.628901 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a26c5-030d-4f8a-a500-1122c89d34ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:27.630570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a26c5-030d-4f8a-a500-1122c89d34ce admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:27.689634 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 945/1876] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:27.692247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a26c5-030d-4f8a-a500-1122c89d34ce admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:27.703235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-be7a26c5-030d-4f8a-a500-1122c89d34ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:27.736403 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a26c5-030d-4f8a-a500-1122c89d34ce admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:27.744304 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc112535-38fa-432a-9c2f-dfdf7ec6f0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:27.745327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc112535-38fa-432a-9c2f-dfdf7ec6f0fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:27.745456 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1902156-3fda-44a8-8f94-0dc3ba0915cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:27.745698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc112535-38fa-432a-9c2f-dfdf7ec6f0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:27.745807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc112535-38fa-432a-9c2f-dfdf7ec6f0fe admin admin] PATH_INFO: `/v3/users/89b164163ed842c1869a2233212a3aa2` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:27.745993 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1902156-3fda-44a8-8f94-0dc3ba0915cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:27.746516 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1902156-3fda-44a8-8f94-0dc3ba0915cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:27.746727 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1902156-3fda-44a8-8f94-0dc3ba0915cc admin admin] PATH_INFO: `/v3/projects/bd796d58aa704ead805b4478cf16e4ee` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:27.748072 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc112535-38fa-432a-9c2f-dfdf7ec6f0fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=89b164163ed842c1869a2233212a3aa2)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:27.748853 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1902156-3fda-44a8-8f94-0dc3ba0915cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd796d58aa704ead805b4478cf16e4ee)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:27.751321 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc112535-38fa-432a-9c2f-dfdf7ec6f0fe admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:27.751483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1902156-3fda-44a8-8f94-0dc3ba0915cc admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:27.797145 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:27.831027 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-dc112535-38fa-432a-9c2f-dfdf7ec6f0fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89b164163ed842c1869a2233212a3aa2'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:27.833691 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-dc112535-38fa-432a-9c2f-dfdf7ec6f0fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89b164163ed842c1869a2233212a3aa2'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:27.854821 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-dc112535-38fa-432a-9c2f-dfdf7ec6f0fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89b164163ed842c1869a2233212a3aa2'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:27.865844 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b1902156-3fda-44a8-8f94-0dc3ba0915cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd796d58aa704ead805b4478cf16e4ee', 'request_id': 'req-b1902156-3fda-44a8-8f94-0dc3ba0915cc'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:27.875471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-dc112535-38fa-432a-9c2f-dfdf7ec6f0fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89b164163ed842c1869a2233212a3aa2'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:27.876432 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-dc112535-38fa-432a-9c2f-dfdf7ec6f0fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89b164163ed842c1869a2233212a3aa2'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:27.878208 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b1902156-3fda-44a8-8f94-0dc3ba0915cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd796d58aa704ead805b4478cf16e4ee', 'request_id': 'req-b1902156-3fda-44a8-8f94-0dc3ba0915cc'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:27.887138 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b1902156-3fda-44a8-8f94-0dc3ba0915cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd796d58aa704ead805b4478cf16e4ee', 'request_id': 'req-b1902156-3fda-44a8-8f94-0dc3ba0915cc'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:27.902621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 932/1877] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:27 2022] DELETE /identity/v3/users/89b164163ed842c1869a2233212a3aa2 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:27.902621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc112535-38fa-432a-9c2f-dfdf7ec6f0fe admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:27.905475 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-dc112535-38fa-432a-9c2f-dfdf7ec6f0fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:27.917646 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b1902156-3fda-44a8-8f94-0dc3ba0915cc admin admin] The token cache is being invalidate because project bd796d58aa704ead805b4478cf16e4ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:27.917945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b1902156-3fda-44a8-8f94-0dc3ba0915cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:27.923717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b1902156-3fda-44a8-8f94-0dc3ba0915cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:27.925582 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 946/1878] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:27 2022] DELETE /identity/v3/projects/bd796d58aa704ead805b4478cf16e4ee => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:27.926990 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74faf0-03ab-412e-bd20-71d08e69bc1c None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:27.927284 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74faf0-03ab-412e-bd20-71d08e69bc1c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:27.927546 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74faf0-03ab-412e-bd20-71d08e69bc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:27.962782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-0a74faf0-03ab-412e-bd20-71d08e69bc1c None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:27.971000 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0a74faf0-03ab-412e-bd20-71d08e69bc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:28.045363 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:28.052212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc112535-38fa-432a-9c2f-dfdf7ec6f0fe admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:28.066810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461ed42-8acf-4f9c-9180-80e392607785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:28.067482 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461ed42-8acf-4f9c-9180-80e392607785 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:28.067774 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461ed42-8acf-4f9c-9180-80e392607785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:28.068049 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461ed42-8acf-4f9c-9180-80e392607785 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:28.068839 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461ed42-8acf-4f9c-9180-80e392607785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:28.072238 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461ed42-8acf-4f9c-9180-80e392607785 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:28.076915 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 947/1879] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:28.082297 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74faf0-03ab-412e-bd20-71d08e69bc1c None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:28.083826 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0a74faf0-03ab-412e-bd20-71d08e69bc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:28.090288 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 933/1880] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:28.090288 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461ed42-8acf-4f9c-9180-80e392607785 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:28.098925 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-8461ed42-8acf-4f9c-9180-80e392607785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:28.106527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74faf0-03ab-412e-bd20-71d08e69bc1c None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:28.112332 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905d18a-fc70-47b1-85fe-fc5eb3c0dfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:28.113003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905d18a-fc70-47b1-85fe-fc5eb3c0dfaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:28.130677 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905d18a-fc70-47b1-85fe-fc5eb3c0dfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:28.131099 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905d18a-fc70-47b1-85fe-fc5eb3c0dfaf admin admin] PATH_INFO: `/v3/users/5f8f9648acec4f7a930f2e662143468e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:28.137177 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905d18a-fc70-47b1-85fe-fc5eb3c0dfaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f8f9648acec4f7a930f2e662143468e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:28.141896 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905d18a-fc70-47b1-85fe-fc5eb3c0dfaf admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:28.228011 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461ed42-8acf-4f9c-9180-80e392607785 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:28.233362 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c905d18a-fc70-47b1-85fe-fc5eb3c0dfaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f8f9648acec4f7a930f2e662143468e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:28.242265 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c905d18a-fc70-47b1-85fe-fc5eb3c0dfaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f8f9648acec4f7a930f2e662143468e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:28.242765 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9efdc3-518f-453b-a2b4-c9026c88bc64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:28.243178 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c905d18a-fc70-47b1-85fe-fc5eb3c0dfaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f8f9648acec4f7a930f2e662143468e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:28.243427 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9efdc3-518f-453b-a2b4-c9026c88bc64 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:28.243707 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9efdc3-518f-453b-a2b4-c9026c88bc64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:28.243975 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9efdc3-518f-453b-a2b4-c9026c88bc64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:28.246138 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-3d9efdc3-518f-453b-a2b4-c9026c88bc64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:28.282693 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9efdc3-518f-453b-a2b4-c9026c88bc64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:28.284170 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9efdc3-518f-453b-a2b4-c9026c88bc64 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:28.301873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c905d18a-fc70-47b1-85fe-fc5eb3c0dfaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f8f9648acec4f7a930f2e662143468e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:28.303255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c905d18a-fc70-47b1-85fe-fc5eb3c0dfaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f8f9648acec4f7a930f2e662143468e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:28.309518 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 934/1881] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:28.311020 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9efdc3-518f-453b-a2b4-c9026c88bc64 service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:28.312720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-3d9efdc3-518f-453b-a2b4-c9026c88bc64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:28.334803 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 948/1882] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:28 2022] DELETE /identity/v3/users/5f8f9648acec4f7a930f2e662143468e => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:28.344752 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9efdc3-518f-453b-a2b4-c9026c88bc64 service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:28.409993 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:28.437299 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6073d-97ef-4559-ad2c-39d1d96cd493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:28.438300 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6073d-97ef-4559-ad2c-39d1d96cd493 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:28.438553 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6073d-97ef-4559-ad2c-39d1d96cd493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:28.438904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6073d-97ef-4559-ad2c-39d1d96cd493 admin admin] PATH_INFO: `/v3/projects/fbe08d3108834be685db29cd44bbf4f1/users/c82326fa82084c45a9a33eb0b4cc4375/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:28.463108 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6073d-97ef-4559-ad2c-39d1d96cd493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbe08d3108834be685db29cd44bbf4f1, user_id=c82326fa82084c45a9a33eb0b4cc4375, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:28.467567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6073d-97ef-4559-ad2c-39d1d96cd493 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:28.488944 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 935/1883] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:28 2022] PUT /identity/v3/projects/fbe08d3108834be685db29cd44bbf4f1/users/c82326fa82084c45a9a33eb0b4cc4375/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:28.502792 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcd8fb-d0a0-459d-bf59-f800400c2ef6 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:28.503010 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcd8fb-d0a0-459d-bf59-f800400c2ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:28.503215 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcd8fb-d0a0-459d-bf59-f800400c2ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:28.547738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-55bcd8fb-d0a0-459d-bf59-f800400c2ef6 None None] MFA Rules not processed for user `c82326fa82084c45a9a33eb0b4cc4375`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:28.548811 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-55bcd8fb-d0a0-459d-bf59-f800400c2ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:28.691655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6073d-97ef-4559-ad2c-39d1d96cd493 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:28.695424 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-e8e6073d-97ef-4559-ad2c-39d1d96cd493 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:28.748749 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 949/1884] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:28.752019 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908541fe-525f-494c-85d7-596bef9d568b None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:28.754476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908541fe-525f-494c-85d7-596bef9d568b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:28.756391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908541fe-525f-494c-85d7-596bef9d568b None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:28.759179 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 950/1885] 10.222.0.47 () {54 vars in 921 bytes} [Thu Aug 4 04:39:28 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:39:28.761194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908541fe-525f-494c-85d7-596bef9d568b None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:28.850028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6073d-97ef-4559-ad2c-39d1d96cd493 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:28.855247 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9421d-ee8f-472d-956f-bbb6a7ea3929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:28.855922 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9421d-ee8f-472d-956f-bbb6a7ea3929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:28.856808 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9421d-ee8f-472d-956f-bbb6a7ea3929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:28.857641 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9421d-ee8f-472d-956f-bbb6a7ea3929 admin admin] PATH_INFO: `/v3/projects/6283c7287061499bbbd70830190ce820` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:28.858705 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9421d-ee8f-472d-956f-bbb6a7ea3929 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6283c7287061499bbbd70830190ce820)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:28.861046 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9421d-ee8f-472d-956f-bbb6a7ea3929 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:28.867542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:28.902244 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908541fe-525f-494c-85d7-596bef9d568b None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:28.909595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b6124-ff0e-4f09-a9cc-5d1a5b32b0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:28.910701 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b6124-ff0e-4f09-a9cc-5d1a5b32b0b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:28.911181 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b6124-ff0e-4f09-a9cc-5d1a5b32b0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:28.911611 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b6124-ff0e-4f09-a9cc-5d1a5b32b0b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:28.912436 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b6124-ff0e-4f09-a9cc-5d1a5b32b0b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:28.915458 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b6124-ff0e-4f09-a9cc-5d1a5b32b0b2 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:28.940798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-7ea9421d-ee8f-472d-956f-bbb6a7ea3929 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6283c7287061499bbbd70830190ce820', 'request_id': 'req-7ea9421d-ee8f-472d-956f-bbb6a7ea3929'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:28.953279 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-7ea9421d-ee8f-472d-956f-bbb6a7ea3929 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6283c7287061499bbbd70830190ce820', 'request_id': 'req-7ea9421d-ee8f-472d-956f-bbb6a7ea3929'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:28.958659 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 951/1886] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:28 2022] POST /identity/v3/projects => generated 379 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:28.960581 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a2fdf-fb84-471c-b485-975c8a063c35 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:28.961318 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a2fdf-fb84-471c-b485-975c8a063c35 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:28.961792 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a2fdf-fb84-471c-b485-975c8a063c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:28.962274 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-7ea9421d-ee8f-472d-956f-bbb6a7ea3929 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6283c7287061499bbbd70830190ce820', 'request_id': 'req-7ea9421d-ee8f-472d-956f-bbb6a7ea3929'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:28.982878 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-7ea9421d-ee8f-472d-956f-bbb6a7ea3929 admin admin] The token cache is being invalidate because project 6283c7287061499bbbd70830190ce820 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:28.983749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-7ea9421d-ee8f-472d-956f-bbb6a7ea3929 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:28.985888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-7ea9421d-ee8f-472d-956f-bbb6a7ea3929 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:28.988967 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 936/1887] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:28 2022] DELETE /identity/v3/projects/6283c7287061499bbbd70830190ce820 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:28.990005 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-967a2fdf-fb84-471c-b485-975c8a063c35 None None] MFA Rules not processed for user `caa2901451aa4554bdfcac1bddbc620a`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:28.990825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9421d-ee8f-472d-956f-bbb6a7ea3929 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:28.991869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-7ea9421d-ee8f-472d-956f-bbb6a7ea3929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:28.992563 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-967a2fdf-fb84-471c-b485-975c8a063c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:29.125779 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:29.133770 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9421d-ee8f-472d-956f-bbb6a7ea3929 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:29.152660 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 952/1888] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Aug 4 04:39:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:29.155588 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08579af3-0c85-4d65-ba40-bf284640415c None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:29.155859 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08579af3-0c85-4d65-ba40-bf284640415c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:29.156129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08579af3-0c85-4d65-ba40-bf284640415c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:29.193572 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-08579af3-0c85-4d65-ba40-bf284640415c None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:29.195551 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-08579af3-0c85-4d65-ba40-bf284640415c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:29.293759 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ce67d-d08d-49ff-95da-abf230d33477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:29.295122 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ce67d-d08d-49ff-95da-abf230d33477 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:29.295604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ce67d-d08d-49ff-95da-abf230d33477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:29.295987 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ce67d-d08d-49ff-95da-abf230d33477 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:29.296841 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ce67d-d08d-49ff-95da-abf230d33477 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:29.299001 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ce67d-d08d-49ff-95da-abf230d33477 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:29.337863 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 953/1889] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:29.337863 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08579af3-0c85-4d65-ba40-bf284640415c None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:29.337863 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-08579af3-0c85-4d65-ba40-bf284640415c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:29.349206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 937/1890] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:28 2022] POST /identity/v3/users => generated 290 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:29.350125 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ce67d-d08d-49ff-95da-abf230d33477 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:29.353680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-4b6ce67d-d08d-49ff-95da-abf230d33477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:29.361994 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08579af3-0c85-4d65-ba40-bf284640415c None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:29.369778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c851a-430a-4f38-a9b1-e48b630538fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:29.370532 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c851a-430a-4f38-a9b1-e48b630538fc service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:29.372000 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c851a-430a-4f38-a9b1-e48b630538fc service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:29.372163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ce67d-d08d-49ff-95da-abf230d33477 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:29.372855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c851a-430a-4f38-a9b1-e48b630538fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:29.384141 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5167395-c38a-491d-9f5c-fd46a9b9547a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:29.384817 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5167395-c38a-491d-9f5c-fd46a9b9547a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:29.385124 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5167395-c38a-491d-9f5c-fd46a9b9547a admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:29.385676 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5167395-c38a-491d-9f5c-fd46a9b9547a admin admin] PATH_INFO: `/v3/projects/d5d6cbe484a7475ea280b15b7a798789` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:29.387581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5167395-c38a-491d-9f5c-fd46a9b9547a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5d6cbe484a7475ea280b15b7a798789)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:29.389728 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5167395-c38a-491d-9f5c-fd46a9b9547a admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:29.396655 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c851a-430a-4f38-a9b1-e48b630538fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:29.398467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c851a-430a-4f38-a9b1-e48b630538fc service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:29.424569 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 954/1891] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:29.426372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a7ac1-fe32-4f65-9fb2-236a8c6773c1 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:29.426768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a7ac1-fe32-4f65-9fb2-236a8c6773c1 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:29.428809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a7ac1-fe32-4f65-9fb2-236a8c6773c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:29.469985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d5167395-c38a-491d-9f5c-fd46a9b9547a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5d6cbe484a7475ea280b15b7a798789', 'request_id': 'req-d5167395-c38a-491d-9f5c-fd46a9b9547a'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:29.475354 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-813a7ac1-fe32-4f65-9fb2-236a8c6773c1 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:29.478277 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-813a7ac1-fe32-4f65-9fb2-236a8c6773c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:29.484955 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d5167395-c38a-491d-9f5c-fd46a9b9547a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5d6cbe484a7475ea280b15b7a798789', 'request_id': 'req-d5167395-c38a-491d-9f5c-fd46a9b9547a'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:29.494898 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d5167395-c38a-491d-9f5c-fd46a9b9547a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5d6cbe484a7475ea280b15b7a798789', 'request_id': 'req-d5167395-c38a-491d-9f5c-fd46a9b9547a'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:29.512673 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d5167395-c38a-491d-9f5c-fd46a9b9547a admin admin] The token cache is being invalidate because project d5d6cbe484a7475ea280b15b7a798789 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:29.513589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d5167395-c38a-491d-9f5c-fd46a9b9547a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:29.514658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d5167395-c38a-491d-9f5c-fd46a9b9547a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:29.516937 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 938/1892] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:29 2022] DELETE /identity/v3/projects/d5d6cbe484a7475ea280b15b7a798789 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:29.520864 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5167395-c38a-491d-9f5c-fd46a9b9547a admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:29.522850 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d5167395-c38a-491d-9f5c-fd46a9b9547a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:29.583144 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:29.612105 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 955/1893] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:29.612105 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a7ac1-fe32-4f65-9fb2-236a8c6773c1 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:29.617391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-813a7ac1-fe32-4f65-9fb2-236a8c6773c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:29.643518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a7ac1-fe32-4f65-9fb2-236a8c6773c1 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:29.653469 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9d1a8-dc23-49e0-9074-8c356cf962e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:29.654456 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9d1a8-dc23-49e0-9074-8c356cf962e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:29.659465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5167395-c38a-491d-9f5c-fd46a9b9547a admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:29.663661 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9d1a8-dc23-49e0-9074-8c356cf962e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:29.664493 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcb060-40de-42b4-8ce7-8acfdb78cc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:29.665140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcb060-40de-42b4-8ce7-8acfdb78cc3d admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:29.665638 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcb060-40de-42b4-8ce7-8acfdb78cc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:29.665915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcb060-40de-42b4-8ce7-8acfdb78cc3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:29.666686 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcb060-40de-42b4-8ce7-8acfdb78cc3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:29.666947 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9d1a8-dc23-49e0-9074-8c356cf962e1 admin admin] PATH_INFO: `/v3/projects/174aedf374c54b5fa33d033e3f34df24` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:29.668502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcb060-40de-42b4-8ce7-8acfdb78cc3d admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:29.672686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff9d1a8-dc23-49e0-9074-8c356cf962e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=174aedf374c54b5fa33d033e3f34df24)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:29.674694 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff9d1a8-dc23-49e0-9074-8c356cf962e1 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:29.684234 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 939/1894] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:29.687503 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcb060-40de-42b4-8ce7-8acfdb78cc3d admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:29.690080 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-a4bcb060-40de-42b4-8ce7-8acfdb78cc3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:29.705577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcb060-40de-42b4-8ce7-8acfdb78cc3d admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:29.712305 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1f87a-8d47-418c-bfef-02e44c3a74c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:29.712984 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1f87a-8d47-418c-bfef-02e44c3a74c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:29.713265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1f87a-8d47-418c-bfef-02e44c3a74c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:29.713680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1f87a-8d47-418c-bfef-02e44c3a74c3 admin admin] PATH_INFO: `/v3/users/62d464ca34534d98858e7e4138a67225` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:29.714809 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1f87a-8d47-418c-bfef-02e44c3a74c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62d464ca34534d98858e7e4138a67225)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:29.717631 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1f87a-8d47-418c-bfef-02e44c3a74c3 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:29.724845 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:29.786739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5ff9d1a8-dc23-49e0-9074-8c356cf962e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '174aedf374c54b5fa33d033e3f34df24', 'request_id': 'req-5ff9d1a8-dc23-49e0-9074-8c356cf962e1'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:29.807352 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5ff9d1a8-dc23-49e0-9074-8c356cf962e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '174aedf374c54b5fa33d033e3f34df24', 'request_id': 'req-5ff9d1a8-dc23-49e0-9074-8c356cf962e1'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:29.816422 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5ff9d1a8-dc23-49e0-9074-8c356cf962e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '174aedf374c54b5fa33d033e3f34df24', 'request_id': 'req-5ff9d1a8-dc23-49e0-9074-8c356cf962e1'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:29.826613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-67b1f87a-8d47-418c-bfef-02e44c3a74c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62d464ca34534d98858e7e4138a67225'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:29.828459 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-67b1f87a-8d47-418c-bfef-02e44c3a74c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62d464ca34534d98858e7e4138a67225'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:29.845881 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-67b1f87a-8d47-418c-bfef-02e44c3a74c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62d464ca34534d98858e7e4138a67225'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:29.862954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5ff9d1a8-dc23-49e0-9074-8c356cf962e1 admin admin] The token cache is being invalidate because project 174aedf374c54b5fa33d033e3f34df24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:29.862954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5ff9d1a8-dc23-49e0-9074-8c356cf962e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:29.863505 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5ff9d1a8-dc23-49e0-9074-8c356cf962e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:29.865957 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 956/1895] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:29 2022] DELETE /identity/v3/projects/174aedf374c54b5fa33d033e3f34df24 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:29.867179 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-67b1f87a-8d47-418c-bfef-02e44c3a74c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62d464ca34534d98858e7e4138a67225'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:29.869994 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-67b1f87a-8d47-418c-bfef-02e44c3a74c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62d464ca34534d98858e7e4138a67225'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:29.874018 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6beb8f-e071-4a05-9324-6db89470b03c None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:29.874366 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6beb8f-e071-4a05-9324-6db89470b03c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:29.874650 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6beb8f-e071-4a05-9324-6db89470b03c None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:29.876921 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 957/1896] 10.222.0.47 () {54 vars in 921 bytes} [Thu Aug 4 04:39:29 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:39:29.877803 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6beb8f-e071-4a05-9324-6db89470b03c None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:29.879580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4f6beb8f-e071-4a05-9324-6db89470b03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:29.915642 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 940/1897] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:29 2022] DELETE /identity/v3/users/62d464ca34534d98858e7e4138a67225 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:29.917040 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1f87a-8d47-418c-bfef-02e44c3a74c3 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:29.919903 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-67b1f87a-8d47-418c-bfef-02e44c3a74c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:29.975161 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6beb8f-e071-4a05-9324-6db89470b03c None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:30.052504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1f87a-8d47-418c-bfef-02e44c3a74c3 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:30.064576 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1b509-f4c0-4806-a6f3-827e0bacbe8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:30.065224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1b509-f4c0-4806-a6f3-827e0bacbe8a admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:30.065516 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1b509-f4c0-4806-a6f3-827e0bacbe8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:30.065832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1b509-f4c0-4806-a6f3-827e0bacbe8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:30.066539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b1b509-f4c0-4806-a6f3-827e0bacbe8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:30.070716 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b1b509-f4c0-4806-a6f3-827e0bacbe8a admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:30.091147 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 941/1898] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:30.094430 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc1a3c-2fa7-48ab-8652-08e5c500beca None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:30.094646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc1a3c-2fa7-48ab-8652-08e5c500beca None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:30.094925 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc1a3c-2fa7-48ab-8652-08e5c500beca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:30.098333 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2effaa-2eac-487c-a82e-c6f9890b42d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:30.099360 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2effaa-2eac-487c-a82e-c6f9890b42d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:30.099360 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2effaa-2eac-487c-a82e-c6f9890b42d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:30.099542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2effaa-2eac-487c-a82e-c6f9890b42d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:30.100305 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2effaa-2eac-487c-a82e-c6f9890b42d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:39:30.100638 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2effaa-2eac-487c-a82e-c6f9890b42d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:30.102414 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2effaa-2eac-487c-a82e-c6f9890b42d6 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:30.117888 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 958/1899] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:39:30.119740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062a6d9-bc6e-4a86-911e-e7a1a4937fb8 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:30.120102 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062a6d9-bc6e-4a86-911e-e7a1a4937fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:30.120434 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062a6d9-bc6e-4a86-911e-e7a1a4937fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:30.133142 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-30dc1a3c-2fa7-48ab-8652-08e5c500beca None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:30.135682 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-30dc1a3c-2fa7-48ab-8652-08e5c500beca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:30.148708 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-f062a6d9-bc6e-4a86-911e-e7a1a4937fb8 None None] MFA Rules not processed for user `caa2901451aa4554bdfcac1bddbc620a`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:30.150172 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f062a6d9-bc6e-4a86-911e-e7a1a4937fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:30.151226 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 942/1900] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:30.152731 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc1a3c-2fa7-48ab-8652-08e5c500beca None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:30.154703 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-30dc1a3c-2fa7-48ab-8652-08e5c500beca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:30.172081 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc1a3c-2fa7-48ab-8652-08e5c500beca None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:30.179470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f08ef-00c2-4e63-890a-e664c339359d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:30.180216 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f08ef-00c2-4e63-890a-e664c339359d admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:30.180515 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f08ef-00c2-4e63-890a-e664c339359d admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:30.181058 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f08ef-00c2-4e63-890a-e664c339359d admin admin] PATH_INFO: `/v3/projects/7f49e0e780c549f598f0770fdcf88d14/users/267ae42789094b3ca13b319a0e54e609/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:30.204207 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f08ef-00c2-4e63-890a-e664c339359d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f49e0e780c549f598f0770fdcf88d14, user_id=267ae42789094b3ca13b319a0e54e609, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:30.207964 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f08ef-00c2-4e63-890a-e664c339359d admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:30.215407 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:30.248368 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 943/1901] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:30 2022] PUT /identity/v3/projects/7f49e0e780c549f598f0770fdcf88d14/users/267ae42789094b3ca13b319a0e54e609/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:30.249692 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f08ef-00c2-4e63-890a-e664c339359d admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:30.366170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 959/1902] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Aug 4 04:39:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:30.366433 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:30.368207 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77ed2f-2672-4dc4-8d0a-26c02c194e9c None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:30.368452 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77ed2f-2672-4dc4-8d0a-26c02c194e9c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:30.368732 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77ed2f-2672-4dc4-8d0a-26c02c194e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:30.393729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f08ef-00c2-4e63-890a-e664c339359d admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:30.399760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-1d77ed2f-2672-4dc4-8d0a-26c02c194e9c None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:30.401002 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1d77ed2f-2672-4dc4-8d0a-26c02c194e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:30.408419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc6d27-1e8e-4960-a2c8-481759814d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:30.409118 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc6d27-1e8e-4960-a2c8-481759814d0a admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:30.409514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc6d27-1e8e-4960-a2c8-481759814d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:30.409899 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc6d27-1e8e-4960-a2c8-481759814d0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:30.414761 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc6d27-1e8e-4960-a2c8-481759814d0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:30.418048 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc6d27-1e8e-4960-a2c8-481759814d0a admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:30.433723 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 960/1903] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:30.436444 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386fefc-0f85-47ea-9d5a-eb437d3a0b9b None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:30.438828 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386fefc-0f85-47ea-9d5a-eb437d3a0b9b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:30.439810 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386fefc-0f85-47ea-9d5a-eb437d3a0b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:30.485789 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-a386fefc-0f85-47ea-9d5a-eb437d3a0b9b None None] MFA Rules not processed for user `267ae42789094b3ca13b319a0e54e609`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:30.488355 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a386fefc-0f85-47ea-9d5a-eb437d3a0b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:30.521701 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 944/1904] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:30 2022] POST /identity/v3/projects => generated 393 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:30.524844 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc6d27-1e8e-4960-a2c8-481759814d0a admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:30.615688 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc6d27-1e8e-4960-a2c8-481759814d0a admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:30.621117 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82d51f-61fd-472e-89f6-ea014441f9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:30.621956 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82d51f-61fd-472e-89f6-ea014441f9aa admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:30.622384 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82d51f-61fd-472e-89f6-ea014441f9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:30.623004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82d51f-61fd-472e-89f6-ea014441f9aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:30.623987 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82d51f-61fd-472e-89f6-ea014441f9aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:39:30.624402 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82d51f-61fd-472e-89f6-ea014441f9aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:30.626525 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82d51f-61fd-472e-89f6-ea014441f9aa admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:30.647382 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 945/1905] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:39:30.647382 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82d51f-61fd-472e-89f6-ea014441f9aa admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:30.649412 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 961/1906] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:30.650461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386fefc-0f85-47ea-9d5a-eb437d3a0b9b None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:30.652724 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-cb82d51f-61fd-472e-89f6-ea014441f9aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:30.669808 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386fefc-0f85-47ea-9d5a-eb437d3a0b9b None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:30.692581 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2217f4-ff17-4dbc-9673-1a01bce4cce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:30.693631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2217f4-ff17-4dbc-9673-1a01bce4cce0 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:30.694055 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2217f4-ff17-4dbc-9673-1a01bce4cce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:30.694464 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2217f4-ff17-4dbc-9673-1a01bce4cce0 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:30.695617 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2217f4-ff17-4dbc-9673-1a01bce4cce0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:30.697761 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2217f4-ff17-4dbc-9673-1a01bce4cce0 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:30.724190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 962/1907] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:30 2022] POST /identity/v3/users => generated 297 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:30.725577 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2217f4-ff17-4dbc-9673-1a01bce4cce0 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:30.741094 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2217f4-ff17-4dbc-9673-1a01bce4cce0 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:30.746970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646b955-d1fc-4b53-b0cf-3fc8e013b82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:30.748276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646b955-d1fc-4b53-b0cf-3fc8e013b82f admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:30.748666 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646b955-d1fc-4b53-b0cf-3fc8e013b82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:30.749062 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646b955-d1fc-4b53-b0cf-3fc8e013b82f admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:30.749891 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646b955-d1fc-4b53-b0cf-3fc8e013b82f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:30.751960 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646b955-d1fc-4b53-b0cf-3fc8e013b82f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:30.777629 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82d51f-61fd-472e-89f6-ea014441f9aa admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:30.783571 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 963/1908] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:30 2022] POST /identity/v3/projects => generated 401 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:30.783971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8454d2-9940-4b03-86f0-a4b24d101434 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:30.784790 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8454d2-9940-4b03-86f0-a4b24d101434 service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:30.785081 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8454d2-9940-4b03-86f0-a4b24d101434 service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:30.785357 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8454d2-9940-4b03-86f0-a4b24d101434 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:30.788070 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646b955-d1fc-4b53-b0cf-3fc8e013b82f admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:30.796634 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8454d2-9940-4b03-86f0-a4b24d101434 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:30.797839 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8454d2-9940-4b03-86f0-a4b24d101434 service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:30.808633 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 946/1909] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:30.810236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8454d2-9940-4b03-86f0-a4b24d101434 service nova] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:30.812161 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-0e8454d2-9940-4b03-86f0-a4b24d101434 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:30.828945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8454d2-9940-4b03-86f0-a4b24d101434 service nova] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:30.862008 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:30.866100 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646b955-d1fc-4b53-b0cf-3fc8e013b82f admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:30.872536 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ac97e-a8cf-4aba-8422-6e8fbb6e2d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:30.873193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ac97e-a8cf-4aba-8422-6e8fbb6e2d1e admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:30.873568 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ac97e-a8cf-4aba-8422-6e8fbb6e2d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:30.873928 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ac97e-a8cf-4aba-8422-6e8fbb6e2d1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:30.874614 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971ac97e-a8cf-4aba-8422-6e8fbb6e2d1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:30.876537 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971ac97e-a8cf-4aba-8422-6e8fbb6e2d1e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:30.891326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:30.905436 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2027f6-4d7d-4315-abfe-1d2898ddd5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:30.906127 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2027f6-4d7d-4315-abfe-1d2898ddd5e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:30.906403 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2027f6-4d7d-4315-abfe-1d2898ddd5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:30.906676 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2027f6-4d7d-4315-abfe-1d2898ddd5e7 admin admin] PATH_INFO: `/v3/projects/f422b8db0ca14359b538c1d6787ff303` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:30.907856 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2027f6-4d7d-4315-abfe-1d2898ddd5e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f422b8db0ca14359b538c1d6787ff303)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:30.909820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2027f6-4d7d-4315-abfe-1d2898ddd5e7 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:30.915512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 964/1910] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:30 2022] POST /identity/v3/projects => generated 379 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:30.915512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ac97e-a8cf-4aba-8422-6e8fbb6e2d1e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:30.970466 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-ce2027f6-4d7d-4315-abfe-1d2898ddd5e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f422b8db0ca14359b538c1d6787ff303', 'request_id': 'req-ce2027f6-4d7d-4315-abfe-1d2898ddd5e7'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:30.984770 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-ce2027f6-4d7d-4315-abfe-1d2898ddd5e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f422b8db0ca14359b538c1d6787ff303', 'request_id': 'req-ce2027f6-4d7d-4315-abfe-1d2898ddd5e7'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:30.998483 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-ce2027f6-4d7d-4315-abfe-1d2898ddd5e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f422b8db0ca14359b538c1d6787ff303', 'request_id': 'req-ce2027f6-4d7d-4315-abfe-1d2898ddd5e7'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:31.006238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ac97e-a8cf-4aba-8422-6e8fbb6e2d1e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:31.013954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-ce2027f6-4d7d-4315-abfe-1d2898ddd5e7 admin admin] The token cache is being invalidate because project f422b8db0ca14359b538c1d6787ff303 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:31.014420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-ce2027f6-4d7d-4315-abfe-1d2898ddd5e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:31.016112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-ce2027f6-4d7d-4315-abfe-1d2898ddd5e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:31.025447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 947/1911] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:30 2022] DELETE /identity/v3/projects/f422b8db0ca14359b538c1d6787ff303 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:31.028177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2027f6-4d7d-4315-abfe-1d2898ddd5e7 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:31.032537 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-ce2027f6-4d7d-4315-abfe-1d2898ddd5e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:31.099408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26febd5a-3908-4b65-9e1b-cbf86bc112ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:31.100110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26febd5a-3908-4b65-9e1b-cbf86bc112ea admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:31.100402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26febd5a-3908-4b65-9e1b-cbf86bc112ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:31.100678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26febd5a-3908-4b65-9e1b-cbf86bc112ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:31.101981 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26febd5a-3908-4b65-9e1b-cbf86bc112ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:31.103901 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26febd5a-3908-4b65-9e1b-cbf86bc112ea admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:31.117351 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 965/1912] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:31.119550 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26febd5a-3908-4b65-9e1b-cbf86bc112ea admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:31.120854 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-26febd5a-3908-4b65-9e1b-cbf86bc112ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:31.141580 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2027f6-4d7d-4315-abfe-1d2898ddd5e7 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:31.141783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26febd5a-3908-4b65-9e1b-cbf86bc112ea admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:31.141888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ae7e3-04ca-4981-af8a-99531bfb6b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:31.141888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ae7e3-04ca-4981-af8a-99531bfb6b1c admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:31.141888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ae7e3-04ca-4981-af8a-99531bfb6b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:31.141888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ae7e3-04ca-4981-af8a-99531bfb6b1c admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:31.142202 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ae7e3-04ca-4981-af8a-99531bfb6b1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:31.142282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5cb29f-e048-4355-8838-05e464b92060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:31.142282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cb29f-e048-4355-8838-05e464b92060 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:31.142282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cb29f-e048-4355-8838-05e464b92060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:31.142282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cb29f-e048-4355-8838-05e464b92060 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:31.143373 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ae7e3-04ca-4981-af8a-99531bfb6b1c admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:31.143495 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5cb29f-e048-4355-8838-05e464b92060 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:31.148345 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5cb29f-e048-4355-8838-05e464b92060 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:31.169501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 966/1913] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:31 2022] POST /identity/v3/users => generated 296 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:31.172200 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a50431-e41a-46a7-b3fa-06a636b2b5b2 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:31.172200 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a50431-e41a-46a7-b3fa-06a636b2b5b2 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:31.172200 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a50431-e41a-46a7-b3fa-06a636b2b5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:31.177286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 948/1914] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:31 2022] POST /identity/v3/users => generated 301 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:31.180937 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ae7e3-04ca-4981-af8a-99531bfb6b1c admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:31.183306 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-a68ae7e3-04ca-4981-af8a-99531bfb6b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:31.195335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-12a50431-e41a-46a7-b3fa-06a636b2b5b2 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:31.196486 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-12a50431-e41a-46a7-b3fa-06a636b2b5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:31.197075 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ae7e3-04ca-4981-af8a-99531bfb6b1c admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:31.202643 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee733ab9-6da2-4609-9ae8-fa1cf1d45953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:31.203464 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee733ab9-6da2-4609-9ae8-fa1cf1d45953 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:31.203754 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee733ab9-6da2-4609-9ae8-fa1cf1d45953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:31.204038 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee733ab9-6da2-4609-9ae8-fa1cf1d45953 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:31.204756 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee733ab9-6da2-4609-9ae8-fa1cf1d45953 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:31.206412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee733ab9-6da2-4609-9ae8-fa1cf1d45953 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:31.207503 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 967/1915] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:31.208448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a50431-e41a-46a7-b3fa-06a636b2b5b2 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:31.218985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 949/1916] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:31.219863 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee733ab9-6da2-4609-9ae8-fa1cf1d45953 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:31.227692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a50431-e41a-46a7-b3fa-06a636b2b5b2 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:31.234255 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee733ab9-6da2-4609-9ae8-fa1cf1d45953 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:31.235209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a20c60-5b38-4006-b210-4f1ec6979992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:31.236343 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a20c60-5b38-4006-b210-4f1ec6979992 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:31.236902 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a20c60-5b38-4006-b210-4f1ec6979992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:31.237466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a20c60-5b38-4006-b210-4f1ec6979992 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:31.238682 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a20c60-5b38-4006-b210-4f1ec6979992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:31.239300 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf579e-8cac-425e-8b58-03bdaca4a2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:31.239994 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf579e-8cac-425e-8b58-03bdaca4a2d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:31.240347 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf579e-8cac-425e-8b58-03bdaca4a2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:31.240693 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf579e-8cac-425e-8b58-03bdaca4a2d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:31.241487 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf579e-8cac-425e-8b58-03bdaca4a2d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:31.241674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a20c60-5b38-4006-b210-4f1ec6979992 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:31.243318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf579e-8cac-425e-8b58-03bdaca4a2d7 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:31.255551 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 950/1917] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:31.256423 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 968/1918] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:31.257056 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6b8df-7523-4bfe-9624-4c38aed95597 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:31.257407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6b8df-7523-4bfe-9624-4c38aed95597 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:31.257767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a20c60-5b38-4006-b210-4f1ec6979992 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:31.257869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6b8df-7523-4bfe-9624-4c38aed95597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:31.270244 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a20c60-5b38-4006-b210-4f1ec6979992 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:31.278442 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71153dff-27a8-4ba8-8e2f-9b19e521af27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:31.280283 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71153dff-27a8-4ba8-8e2f-9b19e521af27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:31.280699 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71153dff-27a8-4ba8-8e2f-9b19e521af27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:31.281299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71153dff-27a8-4ba8-8e2f-9b19e521af27 admin admin] PATH_INFO: `/v3/projects/a002b16665154ebc8128e8c25dde575a/users/996847a47e724b1db64bcc3b66311cce/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:31.284097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-24b6b8df-7523-4bfe-9624-4c38aed95597 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:31.284097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-24b6b8df-7523-4bfe-9624-4c38aed95597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:31.293122 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 951/1919] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:31.293285 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6b8df-7523-4bfe-9624-4c38aed95597 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:31.303872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71153dff-27a8-4ba8-8e2f-9b19e521af27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a002b16665154ebc8128e8c25dde575a, user_id=996847a47e724b1db64bcc3b66311cce, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:31.304033 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6b8df-7523-4bfe-9624-4c38aed95597 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:31.306777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71153dff-27a8-4ba8-8e2f-9b19e521af27 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:31.320445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2328c2-9ee7-4c7b-a154-2466873944f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:31.321244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2328c2-9ee7-4c7b-a154-2466873944f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:31.321530 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2328c2-9ee7-4c7b-a154-2466873944f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:31.321843 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2328c2-9ee7-4c7b-a154-2466873944f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:31.322550 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2328c2-9ee7-4c7b-a154-2466873944f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:31.324133 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2328c2-9ee7-4c7b-a154-2466873944f7 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:31.326912 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 969/1920] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:31 2022] PUT /identity/v3/projects/a002b16665154ebc8128e8c25dde575a/users/996847a47e724b1db64bcc3b66311cce/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:31.330402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71153dff-27a8-4ba8-8e2f-9b19e521af27 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:31.343276 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 952/1921] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:31.343276 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2328c2-9ee7-4c7b-a154-2466873944f7 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:31.350756 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2328c2-9ee7-4c7b-a154-2466873944f7 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:31.368978 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:31.382440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:31.408999 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71153dff-27a8-4ba8-8e2f-9b19e521af27 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:31.413873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f655aa-f40e-4f86-8b18-a63d75b3068f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:31.414621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f655aa-f40e-4f86-8b18-a63d75b3068f admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:31.414901 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f655aa-f40e-4f86-8b18-a63d75b3068f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:31.415174 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f655aa-f40e-4f86-8b18-a63d75b3068f admin admin] PATH_INFO: `/v3/projects/ac4ba2ce6e014c00bf7a2be72d85c977/users/e5d59349482d479c9450c106025850f6/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:31.415980 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149462e-6c34-4cd4-b61f-a8f4fd4b9cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:31.416605 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149462e-6c34-4cd4-b61f-a8f4fd4b9cad admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:31.416888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149462e-6c34-4cd4-b61f-a8f4fd4b9cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:31.417169 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149462e-6c34-4cd4-b61f-a8f4fd4b9cad admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:31.417894 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149462e-6c34-4cd4-b61f-a8f4fd4b9cad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:39:31.418206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149462e-6c34-4cd4-b61f-a8f4fd4b9cad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:31.421753 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149462e-6c34-4cd4-b61f-a8f4fd4b9cad admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:31.431107 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f655aa-f40e-4f86-8b18-a63d75b3068f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac4ba2ce6e014c00bf7a2be72d85c977, user_id=e5d59349482d479c9450c106025850f6, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:31.432652 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 953/1922] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:39:31.433812 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f655aa-f40e-4f86-8b18-a63d75b3068f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:31.434052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24948db3-5bc6-4466-b31e-e6d7ba4232cd None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:31.434343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24948db3-5bc6-4466-b31e-e6d7ba4232cd None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:31.435148 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24948db3-5bc6-4466-b31e-e6d7ba4232cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:31.453322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 970/1923] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:31 2022] PUT /identity/v3/projects/ac4ba2ce6e014c00bf7a2be72d85c977/users/e5d59349482d479c9450c106025850f6/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:31.453322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f655aa-f40e-4f86-8b18-a63d75b3068f admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:31.468391 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-24948db3-5bc6-4466-b31e-e6d7ba4232cd None None] MFA Rules not processed for user `996847a47e724b1db64bcc3b66311cce`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:31.469623 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-24948db3-5bc6-4466-b31e-e6d7ba4232cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:31.522827 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f655aa-f40e-4f86-8b18-a63d75b3068f admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:31.527467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99917153-270a-4031-94d8-a1658e4cc76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:31.528302 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99917153-270a-4031-94d8-a1658e4cc76e admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:31.528814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99917153-270a-4031-94d8-a1658e4cc76e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:31.529267 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99917153-270a-4031-94d8-a1658e4cc76e admin admin] PATH_INFO: `/v3/projects/f30ffab061894599b9e22de09a9c6623/users/dd2bed06bc264a44bc61421c0f5afb26/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:31.546316 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 954/1924] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:31.546316 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24948db3-5bc6-4466-b31e-e6d7ba4232cd None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:31.556432 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99917153-270a-4031-94d8-a1658e4cc76e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f30ffab061894599b9e22de09a9c6623, user_id=dd2bed06bc264a44bc61421c0f5afb26, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:31.559331 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99917153-270a-4031-94d8-a1658e4cc76e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:31.560362 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24948db3-5bc6-4466-b31e-e6d7ba4232cd None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:31.570109 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42789192-a0fc-4474-91f6-a10d1c5ecd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:31.570964 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42789192-a0fc-4474-91f6-a10d1c5ecd07 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:31.571345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42789192-a0fc-4474-91f6-a10d1c5ecd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:31.571709 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42789192-a0fc-4474-91f6-a10d1c5ecd07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:31.573783 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42789192-a0fc-4474-91f6-a10d1c5ecd07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:31.574669 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42789192-a0fc-4474-91f6-a10d1c5ecd07 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:31.582206 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 971/1925] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:31 2022] PUT /identity/v3/projects/f30ffab061894599b9e22de09a9c6623/users/dd2bed06bc264a44bc61421c0f5afb26/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:31.582988 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99917153-270a-4031-94d8-a1658e4cc76e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:31.612270 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 955/1926] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:31 2022] POST /identity/v3/projects => generated 375 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:31.613123 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42789192-a0fc-4474-91f6-a10d1c5ecd07 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:31.674597 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99917153-270a-4031-94d8-a1658e4cc76e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:31.739812 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:31.755559 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42789192-a0fc-4474-91f6-a10d1c5ecd07 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:31.769970 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7b6e1-6412-46f5-885b-ac3dd51e6bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:31.771111 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7b6e1-6412-46f5-885b-ac3dd51e6bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:31.771518 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7b6e1-6412-46f5-885b-ac3dd51e6bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:31.771968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7b6e1-6412-46f5-885b-ac3dd51e6bb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:31.772949 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c7b6e1-6412-46f5-885b-ac3dd51e6bb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:31.775955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81b5a6-d6e5-41a8-90b3-840b9307c502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:31.776758 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81b5a6-d6e5-41a8-90b3-840b9307c502 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:31.777092 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81b5a6-d6e5-41a8-90b3-840b9307c502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:31.777368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81b5a6-d6e5-41a8-90b3-840b9307c502 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:31.778113 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb81b5a6-d6e5-41a8-90b3-840b9307c502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:31.779978 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb81b5a6-d6e5-41a8-90b3-840b9307c502 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:31.781704 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c7b6e1-6412-46f5-885b-ac3dd51e6bb0 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:31.794952 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 972/1927] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:31.802302 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723b1b1-cc7c-4fb4-9e81-1e00e63e732e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:31.802608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723b1b1-cc7c-4fb4-9e81-1e00e63e732e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:31.802871 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723b1b1-cc7c-4fb4-9e81-1e00e63e732e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:31.834205 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 956/1928] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:31 2022] POST /identity/v3/projects => generated 391 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:31.834994 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7b6e1-6412-46f5-885b-ac3dd51e6bb0 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:31.845765 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-3723b1b1-cc7c-4fb4-9e81-1e00e63e732e None None] MFA Rules not processed for user `dd2bed06bc264a44bc61421c0f5afb26`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:31.847383 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3723b1b1-cc7c-4fb4-9e81-1e00e63e732e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:31.869177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:31.917527 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7b6e1-6412-46f5-885b-ac3dd51e6bb0 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:31.923285 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890ccc7-bb95-4e2a-82ff-3f948c5e82f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:31.924039 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890ccc7-bb95-4e2a-82ff-3f948c5e82f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:31.924209 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890ccc7-bb95-4e2a-82ff-3f948c5e82f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:31.924483 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890ccc7-bb95-4e2a-82ff-3f948c5e82f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:31.925241 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890ccc7-bb95-4e2a-82ff-3f948c5e82f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:31.927352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890ccc7-bb95-4e2a-82ff-3f948c5e82f7 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:31.936291 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 973/1929] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:31 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:31.937342 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3723b1b1-cc7c-4fb4-9e81-1e00e63e732e None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:31.948551 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3723b1b1-cc7c-4fb4-9e81-1e00e63e732e None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:31.957367 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a68ace-c55e-45e3-8f42-a01d7911c581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:31.958301 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a68ace-c55e-45e3-8f42-a01d7911c581 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:31.958750 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a68ace-c55e-45e3-8f42-a01d7911c581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:31.959141 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a68ace-c55e-45e3-8f42-a01d7911c581 admin admin] PATH_INFO: `/v3/projects/ac4ba2ce6e014c00bf7a2be72d85c977/users/e5d59349482d479c9450c106025850f6/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:31.961278 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a68ace-c55e-45e3-8f42-a01d7911c581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac4ba2ce6e014c00bf7a2be72d85c977, user_id=e5d59349482d479c9450c106025850f6, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:31.964386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a68ace-c55e-45e3-8f42-a01d7911c581 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:31.984124 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 957/1930] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:31 2022] POST /identity/v3/users => generated 288 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:31.984124 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890ccc7-bb95-4e2a-82ff-3f948c5e82f7 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:31.989701 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 974/1931] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:31 2022] PUT /identity/v3/projects/ac4ba2ce6e014c00bf7a2be72d85c977/users/e5d59349482d479c9450c106025850f6/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:31.991538 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a68ace-c55e-45e3-8f42-a01d7911c581 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:32.008700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890ccc7-bb95-4e2a-82ff-3f948c5e82f7 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:32.078095 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a68ace-c55e-45e3-8f42-a01d7911c581 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:32.079955 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105342c0-9fc3-4e70-b297-1a34629319db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:32.080590 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105342c0-9fc3-4e70-b297-1a34629319db admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:32.080869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105342c0-9fc3-4e70-b297-1a34629319db admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:32.081133 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105342c0-9fc3-4e70-b297-1a34629319db admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:32.081840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105342c0-9fc3-4e70-b297-1a34629319db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:32.083627 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4c6e3-704f-430b-96e2-a999e91671b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:32.083843 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105342c0-9fc3-4e70-b297-1a34629319db admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:32.084834 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4c6e3-704f-430b-96e2-a999e91671b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:32.084834 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4c6e3-704f-430b-96e2-a999e91671b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:32.085242 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4c6e3-704f-430b-96e2-a999e91671b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:32.086056 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec4c6e3-704f-430b-96e2-a999e91671b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:32.088109 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec4c6e3-704f-430b-96e2-a999e91671b0 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:32.111704 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 958/1932] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:31 2022] POST /identity/v3/users => generated 302 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:32.115497 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105342c0-9fc3-4e70-b297-1a34629319db admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:32.125929 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 975/1933] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:31 2022] POST /identity/v3/projects => generated 399 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:32.125929 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d240c-2531-4d93-ad24-e593ff6b71f1 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:32.125929 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d240c-2531-4d93-ad24-e593ff6b71f1 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:32.125929 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d240c-2531-4d93-ad24-e593ff6b71f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:32.127325 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105342c0-9fc3-4e70-b297-1a34629319db admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:32.180411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-2d0d240c-2531-4d93-ad24-e593ff6b71f1 None None] MFA Rules not processed for user `e5d59349482d479c9450c106025850f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:32.183062 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2d0d240c-2531-4d93-ad24-e593ff6b71f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:32.208272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8d607-c5fd-4ee1-bde0-a3ddf015eea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:32.208941 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8d607-c5fd-4ee1-bde0-a3ddf015eea3 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:32.209289 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8d607-c5fd-4ee1-bde0-a3ddf015eea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:32.209663 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8d607-c5fd-4ee1-bde0-a3ddf015eea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:32.210456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa8d607-c5fd-4ee1-bde0-a3ddf015eea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:32.212170 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa8d607-c5fd-4ee1-bde0-a3ddf015eea3 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:32.220482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:32.224924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 959/1934] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:32.225840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8d607-c5fd-4ee1-bde0-a3ddf015eea3 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:32.238595 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8d607-c5fd-4ee1-bde0-a3ddf015eea3 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:32.247052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e3cfa-7179-45ff-9933-be8b5136554a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:32.248122 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e3cfa-7179-45ff-9933-be8b5136554a admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:32.248407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e3cfa-7179-45ff-9933-be8b5136554a admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:32.248685 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e3cfa-7179-45ff-9933-be8b5136554a admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:32.249407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e3cfa-7179-45ff-9933-be8b5136554a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:32.251048 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e3cfa-7179-45ff-9933-be8b5136554a admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:32.272637 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 960/1935] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:32.273524 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e3cfa-7179-45ff-9933-be8b5136554a admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:32.273991 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 976/1936] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:32 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:32.275072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d240c-2531-4d93-ad24-e593ff6b71f1 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:32.290054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d240c-2531-4d93-ad24-e593ff6b71f1 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:32.297052 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce4330-e450-4191-afe2-71e48f123188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:32.297878 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce4330-e450-4191-afe2-71e48f123188 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:32.298410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce4330-e450-4191-afe2-71e48f123188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:32.298810 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce4330-e450-4191-afe2-71e48f123188 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:32.299958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce4330-e450-4191-afe2-71e48f123188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:32.300209 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e3cfa-7179-45ff-9933-be8b5136554a admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:32.301510 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce4330-e450-4191-afe2-71e48f123188 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:32.309931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5b13a-2e3c-4fd7-9a4e-88d45d9f90c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:32.311537 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5b13a-2e3c-4fd7-9a4e-88d45d9f90c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:32.311537 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5b13a-2e3c-4fd7-9a4e-88d45d9f90c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:32.312000 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5b13a-2e3c-4fd7-9a4e-88d45d9f90c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:32.312819 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5b13a-2e3c-4fd7-9a4e-88d45d9f90c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:32.315172 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5b13a-2e3c-4fd7-9a4e-88d45d9f90c5 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:32.317727 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 977/1937] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:32.318795 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce4330-e450-4191-afe2-71e48f123188 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:32.331549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce4330-e450-4191-afe2-71e48f123188 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:32.337480 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26679c-ff3c-4db2-a9b7-dc4daab0a074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:32.338210 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26679c-ff3c-4db2-a9b7-dc4daab0a074 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:32.338534 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26679c-ff3c-4db2-a9b7-dc4daab0a074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:32.338812 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26679c-ff3c-4db2-a9b7-dc4daab0a074 admin admin] PATH_INFO: `/v3/projects/86d7ea8d671b49b0b33b3539474649e8/users/a11ba8dee55b4457a043baf7b91dfa9e/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:32.366726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26679c-ff3c-4db2-a9b7-dc4daab0a074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86d7ea8d671b49b0b33b3539474649e8, user_id=a11ba8dee55b4457a043baf7b91dfa9e, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:32.370524 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26679c-ff3c-4db2-a9b7-dc4daab0a074 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:32.376171 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 961/1938] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:32 2022] POST /identity/v3/users => generated 306 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:32.377366 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5b13a-2e3c-4fd7-9a4e-88d45d9f90c5 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:32.382986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-e2e5b13a-2e3c-4fd7-9a4e-88d45d9f90c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:32.392533 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 978/1939] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:32 2022] PUT /identity/v3/projects/86d7ea8d671b49b0b33b3539474649e8/users/a11ba8dee55b4457a043baf7b91dfa9e/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:32.394219 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26679c-ff3c-4db2-a9b7-dc4daab0a074 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:32.421838 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:32.496466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26679c-ff3c-4db2-a9b7-dc4daab0a074 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:32.502471 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9bb470-0198-43bd-b1f4-7e41406019a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:32.503384 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bb470-0198-43bd-b1f4-7e41406019a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:32.503753 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bb470-0198-43bd-b1f4-7e41406019a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:32.504181 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bb470-0198-43bd-b1f4-7e41406019a4 admin admin] PATH_INFO: `/v3/projects/5c82bebd392e41348bfb17685328d8c4/users/dfec8c45a2494b67b81a2605e21f62e5/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:32.522009 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9bb470-0198-43bd-b1f4-7e41406019a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c82bebd392e41348bfb17685328d8c4, user_id=dfec8c45a2494b67b81a2605e21f62e5, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:32.524986 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9bb470-0198-43bd-b1f4-7e41406019a4 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:32.546646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5b13a-2e3c-4fd7-9a4e-88d45d9f90c5 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:32.562024 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 979/1940] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:32 2022] PUT /identity/v3/projects/5c82bebd392e41348bfb17685328d8c4/users/dfec8c45a2494b67b81a2605e21f62e5/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:32.563401 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9bb470-0198-43bd-b1f4-7e41406019a4 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:32.634591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c34d6-0a3c-4d6d-8133-a702b249d9af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:32.635841 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c34d6-0a3c-4d6d-8133-a702b249d9af service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:32.636265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c34d6-0a3c-4d6d-8133-a702b249d9af service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:32.637283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c34d6-0a3c-4d6d-8133-a702b249d9af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:32.654185 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9bb470-0198-43bd-b1f4-7e41406019a4 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:32.660207 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08df343-ec1b-4d7f-9eeb-b940e194b63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:32.660962 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08df343-ec1b-4d7f-9eeb-b940e194b63b admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:32.661458 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08df343-ec1b-4d7f-9eeb-b940e194b63b admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:32.661833 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08df343-ec1b-4d7f-9eeb-b940e194b63b admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:32.662618 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08df343-ec1b-4d7f-9eeb-b940e194b63b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:32.664339 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08df343-ec1b-4d7f-9eeb-b940e194b63b admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:32.677008 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 980/1941] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:32.681778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08df343-ec1b-4d7f-9eeb-b940e194b63b admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:32.698847 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08df343-ec1b-4d7f-9eeb-b940e194b63b admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:32.708200 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162b48e-c62e-4d01-850d-a60b2c32bf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:32.708840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162b48e-c62e-4d01-850d-a60b2c32bf26 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:32.709109 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162b48e-c62e-4d01-850d-a60b2c32bf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:32.709385 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162b48e-c62e-4d01-850d-a60b2c32bf26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:32.710461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162b48e-c62e-4d01-850d-a60b2c32bf26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:32.711839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162b48e-c62e-4d01-850d-a60b2c32bf26 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:32.714642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:32.726198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 981/1942] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:32.732440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b170ba3-35fc-4e16-996f-4be4eb2a0e4b None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:32.733184 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b170ba3-35fc-4e16-996f-4be4eb2a0e4b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:32.733562 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b170ba3-35fc-4e16-996f-4be4eb2a0e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:32.739600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c34d6-0a3c-4d6d-8133-a702b249d9af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:32.741105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c34d6-0a3c-4d6d-8133-a702b249d9af service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:32.752192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 962/1943] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:32 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:32.753090 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c34d6-0a3c-4d6d-8133-a702b249d9af service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:32.765249 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c34d6-0a3c-4d6d-8133-a702b249d9af service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:32.770406 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69c679-1062-4271-8e09-3ef7747254d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:32.771153 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69c679-1062-4271-8e09-3ef7747254d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:32.771496 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69c679-1062-4271-8e09-3ef7747254d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:32.771839 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69c679-1062-4271-8e09-3ef7747254d4 admin admin] PATH_INFO: `/v3/projects/30081bed396a41dda6e1f9c8d1ed09ed/users/a1f4ce5ac5e64740bdb00f279fa16c84/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:32.786474 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7b170ba3-35fc-4e16-996f-4be4eb2a0e4b None None] MFA Rules not processed for user `dfec8c45a2494b67b81a2605e21f62e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:32.788454 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7b170ba3-35fc-4e16-996f-4be4eb2a0e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:32.793956 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69c679-1062-4271-8e09-3ef7747254d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30081bed396a41dda6e1f9c8d1ed09ed, user_id=a1f4ce5ac5e64740bdb00f279fa16c84, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:32.797524 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69c679-1062-4271-8e09-3ef7747254d4 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:32.832345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 963/1944] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:32 2022] PUT /identity/v3/projects/30081bed396a41dda6e1f9c8d1ed09ed/users/a1f4ce5ac5e64740bdb00f279fa16c84/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:32.833590 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69c679-1062-4271-8e09-3ef7747254d4 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:32.964691 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 982/1945] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:32.964691 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b170ba3-35fc-4e16-996f-4be4eb2a0e4b None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:32.984691 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b170ba3-35fc-4e16-996f-4be4eb2a0e4b None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:32.990198 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:33.031054 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69c679-1062-4271-8e09-3ef7747254d4 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:33.060520 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a3e09-8087-41af-9113-1851be844cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:33.061248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a3e09-8087-41af-9113-1851be844cee admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:33.061582 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a3e09-8087-41af-9113-1851be844cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:33.061889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a3e09-8087-41af-9113-1851be844cee admin admin] PATH_INFO: `/v3/projects/86d7ea8d671b49b0b33b3539474649e8/users/a11ba8dee55b4457a043baf7b91dfa9e/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:33.064004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a3e09-8087-41af-9113-1851be844cee admin admin] RBAC: Authorizing `identity:create_grant(project_id=86d7ea8d671b49b0b33b3539474649e8, user_id=a11ba8dee55b4457a043baf7b91dfa9e, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:33.066797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a3e09-8087-41af-9113-1851be844cee admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:33.099198 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 964/1946] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:32 2022] PUT /identity/v3/projects/86d7ea8d671b49b0b33b3539474649e8/users/a11ba8dee55b4457a043baf7b91dfa9e/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:33.110020 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a3e09-8087-41af-9113-1851be844cee admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:33.112730 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-ed2a3e09-8087-41af-9113-1851be844cee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:33.210482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d6214-9ad6-42b7-8b87-303f3beacfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:33.216685 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d6214-9ad6-42b7-8b87-303f3beacfa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:33.218531 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a3e09-8087-41af-9113-1851be844cee admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:33.218826 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d6214-9ad6-42b7-8b87-303f3beacfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:33.222287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d6214-9ad6-42b7-8b87-303f3beacfa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:33.224290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a45a12-0608-49b9-8955-47fb5ff2d385 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:33.224567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d6214-9ad6-42b7-8b87-303f3beacfa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:33.225319 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a45a12-0608-49b9-8955-47fb5ff2d385 service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:33.225829 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a45a12-0608-49b9-8955-47fb5ff2d385 service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:33.226000 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a45a12-0608-49b9-8955-47fb5ff2d385 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:33.228925 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-94a45a12-0608-49b9-8955-47fb5ff2d385 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:33.234862 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d6214-9ad6-42b7-8b87-303f3beacfa3 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:33.264652 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 983/1947] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:33.266242 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901d1cc-7f5b-467c-b5d7-78d0b7bf1c38 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:33.266538 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901d1cc-7f5b-467c-b5d7-78d0b7bf1c38 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:33.266819 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901d1cc-7f5b-467c-b5d7-78d0b7bf1c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:33.336387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-6901d1cc-7f5b-467c-b5d7-78d0b7bf1c38 None None] MFA Rules not processed for user `a11ba8dee55b4457a043baf7b91dfa9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:33.337566 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6901d1cc-7f5b-467c-b5d7-78d0b7bf1c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:33.346059 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a45a12-0608-49b9-8955-47fb5ff2d385 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:33.348158 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a45a12-0608-49b9-8955-47fb5ff2d385 service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:33.366421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 965/1948] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:33.367350 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a45a12-0608-49b9-8955-47fb5ff2d385 service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:33.379284 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a45a12-0608-49b9-8955-47fb5ff2d385 service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:33.385438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480042c1-61ec-4c87-ada4-a3cb15fa9852 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:33.386428 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480042c1-61ec-4c87-ada4-a3cb15fa9852 service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:33.386974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480042c1-61ec-4c87-ada4-a3cb15fa9852 service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:33.387205 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480042c1-61ec-4c87-ada4-a3cb15fa9852 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:33.526413 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 984/1949] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:33 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:33.527908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901d1cc-7f5b-467c-b5d7-78d0b7bf1c38 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:33.542082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:33.551798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480042c1-61ec-4c87-ada4-a3cb15fa9852 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:33.553570 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480042c1-61ec-4c87-ada4-a3cb15fa9852 service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:33.565709 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 966/1950] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:33.662922 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901d1cc-7f5b-467c-b5d7-78d0b7bf1c38 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:33.671319 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9648d9-27fe-4602-8e04-49d43cae35bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:33.672779 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9648d9-27fe-4602-8e04-49d43cae35bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:33.673071 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9648d9-27fe-4602-8e04-49d43cae35bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:33.673351 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9648d9-27fe-4602-8e04-49d43cae35bb admin admin] PATH_INFO: `/v3/projects/30081bed396a41dda6e1f9c8d1ed09ed/users/a1f4ce5ac5e64740bdb00f279fa16c84/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:33.676251 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9648d9-27fe-4602-8e04-49d43cae35bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=30081bed396a41dda6e1f9c8d1ed09ed, user_id=a1f4ce5ac5e64740bdb00f279fa16c84, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:33.679593 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9648d9-27fe-4602-8e04-49d43cae35bb admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:33.703490 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 985/1951] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:33 2022] PUT /identity/v3/projects/30081bed396a41dda6e1f9c8d1ed09ed/users/a1f4ce5ac5e64740bdb00f279fa16c84/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:33.714644 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f234b-6b7b-45b6-987e-01360ca85bd5 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:33.715193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f234b-6b7b-45b6-987e-01360ca85bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:33.715671 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f234b-6b7b-45b6-987e-01360ca85bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:33.799323 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-472f234b-6b7b-45b6-987e-01360ca85bd5 None None] MFA Rules not processed for user `a1f4ce5ac5e64740bdb00f279fa16c84`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:33.805831 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-472f234b-6b7b-45b6-987e-01360ca85bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:33.855142 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:33.891744 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9648d9-27fe-4602-8e04-49d43cae35bb admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:33.974271 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 967/1952] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:33 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:33.984925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9648d9-27fe-4602-8e04-49d43cae35bb admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:33.990679 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1ca8d-428b-4695-a025-bf4a22ac701f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:33.991325 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1ca8d-428b-4695-a025-bf4a22ac701f service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:33.991653 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1ca8d-428b-4695-a025-bf4a22ac701f service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:33.991935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1ca8d-428b-4695-a025-bf4a22ac701f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:34.109963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b1ca8d-428b-4695-a025-bf4a22ac701f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:34.121742 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b1ca8d-428b-4695-a025-bf4a22ac701f service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:34.126672 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 986/1953] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:33 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:39:34.187354 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f234b-6b7b-45b6-987e-01360ca85bd5 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:34.198276 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f234b-6b7b-45b6-987e-01360ca85bd5 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:34.203369 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07b63b-efc3-4c92-b368-ee31d0066ba2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:34.204080 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07b63b-efc3-4c92-b368-ee31d0066ba2 service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:34.204415 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07b63b-efc3-4c92-b368-ee31d0066ba2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:34.205134 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07b63b-efc3-4c92-b368-ee31d0066ba2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:34.217589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e07b63b-efc3-4c92-b368-ee31d0066ba2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:34.217589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e07b63b-efc3-4c92-b368-ee31d0066ba2 service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:34.229249 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 968/1954] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:34 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:34.273572 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1ca8d-428b-4695-a025-bf4a22ac701f service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:34.282605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1ca8d-428b-4695-a025-bf4a22ac701f service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:34.288244 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31381b-3d8e-4c27-9075-2136f3b33f49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:34.289895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31381b-3d8e-4c27-9075-2136f3b33f49 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:34.289895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31381b-3d8e-4c27-9075-2136f3b33f49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:34.289895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31381b-3d8e-4c27-9075-2136f3b33f49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:34.351596 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:34.363991 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31381b-3d8e-4c27-9075-2136f3b33f49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:34.365758 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31381b-3d8e-4c27-9075-2136f3b33f49 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:34.377573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 987/1955] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:34 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:34.720208 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07b63b-efc3-4c92-b368-ee31d0066ba2 service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:34.729793 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07b63b-efc3-4c92-b368-ee31d0066ba2 service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:34.734771 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d0494-c2e3-44c0-a31a-6c1c3630fe74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:34.735401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d0494-c2e3-44c0-a31a-6c1c3630fe74 service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:34.735686 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d0494-c2e3-44c0-a31a-6c1c3630fe74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:34.735966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d0494-c2e3-44c0-a31a-6c1c3630fe74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:34.820247 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d0494-c2e3-44c0-a31a-6c1c3630fe74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:34.821711 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d0494-c2e3-44c0-a31a-6c1c3630fe74 service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:34.831631 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 969/1956] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:34 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:41.695125 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31381b-3d8e-4c27-9075-2136f3b33f49 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:41.800748 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31381b-3d8e-4c27-9075-2136f3b33f49 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:41.805854 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6643a9-f60d-4bda-b146-6c00d1852aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:41.806552 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6643a9-f60d-4bda-b146-6c00d1852aca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:41.806996 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6643a9-f60d-4bda-b146-6c00d1852aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:41.807155 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6643a9-f60d-4bda-b146-6c00d1852aca admin admin] PATH_INFO: `/v3/users/dfec8c45a2494b67b81a2605e21f62e5` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:41.808537 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6643a9-f60d-4bda-b146-6c00d1852aca admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfec8c45a2494b67b81a2605e21f62e5)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:41.811446 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6643a9-f60d-4bda-b146-6c00d1852aca admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:41.879604 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6a6643a9-f60d-4bda-b146-6c00d1852aca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfec8c45a2494b67b81a2605e21f62e5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:41.887812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d0494-c2e3-44c0-a31a-6c1c3630fe74 service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:41.892395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6a6643a9-f60d-4bda-b146-6c00d1852aca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfec8c45a2494b67b81a2605e21f62e5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:41.895029 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6a6643a9-f60d-4bda-b146-6c00d1852aca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfec8c45a2494b67b81a2605e21f62e5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:41.901624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d0494-c2e3-44c0-a31a-6c1c3630fe74 service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:41.912581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b73e85-5c02-403a-8580-3a23675c7ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:41.913488 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6a6643a9-f60d-4bda-b146-6c00d1852aca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfec8c45a2494b67b81a2605e21f62e5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:41.914261 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b73e85-5c02-403a-8580-3a23675c7ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:41.914373 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6a6643a9-f60d-4bda-b146-6c00d1852aca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfec8c45a2494b67b81a2605e21f62e5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:41.915264 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b73e85-5c02-403a-8580-3a23675c7ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:41.916749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b73e85-5c02-403a-8580-3a23675c7ef7 admin admin] PATH_INFO: `/v3/users/996847a47e724b1db64bcc3b66311cce` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:41.918490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b73e85-5c02-403a-8580-3a23675c7ef7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=996847a47e724b1db64bcc3b66311cce)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:41.920715 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b73e85-5c02-403a-8580-3a23675c7ef7 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:41.938031 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 988/1957] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:41 2022] DELETE /identity/v3/users/dfec8c45a2494b67b81a2605e21f62e5 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:41.942789 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612802c-7de0-41ee-b99a-561a5ca0422e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:41.943078 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612802c-7de0-41ee-b99a-561a5ca0422e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:41.943356 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612802c-7de0-41ee-b99a-561a5ca0422e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:41.968662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-a612802c-7de0-41ee-b99a-561a5ca0422e None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:41.970202 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a612802c-7de0-41ee-b99a-561a5ca0422e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:41.990770 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-b2b73e85-5c02-403a-8580-3a23675c7ef7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '996847a47e724b1db64bcc3b66311cce'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:41.991517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-b2b73e85-5c02-403a-8580-3a23675c7ef7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '996847a47e724b1db64bcc3b66311cce'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:42.008142 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-b2b73e85-5c02-403a-8580-3a23675c7ef7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '996847a47e724b1db64bcc3b66311cce'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:42.016598 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-b2b73e85-5c02-403a-8580-3a23675c7ef7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '996847a47e724b1db64bcc3b66311cce'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:42.017409 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-b2b73e85-5c02-403a-8580-3a23675c7ef7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '996847a47e724b1db64bcc3b66311cce'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:42.031464 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 970/1958] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:41 2022] DELETE /identity/v3/users/996847a47e724b1db64bcc3b66311cce => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:39:42.032445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b73e85-5c02-403a-8580-3a23675c7ef7 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:42.034141 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-b2b73e85-5c02-403a-8580-3a23675c7ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:42.054689 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 989/1959] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:39:42.056899 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12c129-5f19-4b93-b48b-141ed42b15ce None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:42.057196 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12c129-5f19-4b93-b48b-141ed42b15ce None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:42.057477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12c129-5f19-4b93-b48b-141ed42b15ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:42.068333 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:42.086736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-fa12c129-5f19-4b93-b48b-141ed42b15ce None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:42.088063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fa12c129-5f19-4b93-b48b-141ed42b15ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:42.093476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:42.124273 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b73e85-5c02-403a-8580-3a23675c7ef7 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:42.130496 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff0dd5-9172-46ba-9317-a032e2c510d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:42.131139 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff0dd5-9172-46ba-9317-a032e2c510d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:42.131540 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff0dd5-9172-46ba-9317-a032e2c510d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:42.131822 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff0dd5-9172-46ba-9317-a032e2c510d5 admin admin] PATH_INFO: `/v3/users/dd2bed06bc264a44bc61421c0f5afb26` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:42.133087 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff0dd5-9172-46ba-9317-a032e2c510d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd2bed06bc264a44bc61421c0f5afb26)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:42.136020 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff0dd5-9172-46ba-9317-a032e2c510d5 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:42.170002 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 990/1960] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:42.171580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa12c129-5f19-4b93-b48b-141ed42b15ce None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:42.172961 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fa12c129-5f19-4b93-b48b-141ed42b15ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:42.201388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c1ff0dd5-9172-46ba-9317-a032e2c510d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd2bed06bc264a44bc61421c0f5afb26'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:42.202336 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c1ff0dd5-9172-46ba-9317-a032e2c510d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd2bed06bc264a44bc61421c0f5afb26'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:42.224724 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c1ff0dd5-9172-46ba-9317-a032e2c510d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd2bed06bc264a44bc61421c0f5afb26'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:42.235212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c1ff0dd5-9172-46ba-9317-a032e2c510d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd2bed06bc264a44bc61421c0f5afb26'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:42.236210 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c1ff0dd5-9172-46ba-9317-a032e2c510d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd2bed06bc264a44bc61421c0f5afb26'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:42.252177 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa12c129-5f19-4b93-b48b-141ed42b15ce None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:42.255945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ddc63-2948-422e-ba24-5131377e9c0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:42.256559 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ddc63-2948-422e-ba24-5131377e9c0a service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:42.256843 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ddc63-2948-422e-ba24-5131377e9c0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:42.257116 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ddc63-2948-422e-ba24-5131377e9c0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:42.260576 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 971/1961] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:42 2022] DELETE /identity/v3/users/dd2bed06bc264a44bc61421c0f5afb26 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:42.260576 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff0dd5-9172-46ba-9317-a032e2c510d5 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:42.263376 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c1ff0dd5-9172-46ba-9317-a032e2c510d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:42.263611 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3c5ddc63-2948-422e-ba24-5131377e9c0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:42.278375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ddc63-2948-422e-ba24-5131377e9c0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:42.280280 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ddc63-2948-422e-ba24-5131377e9c0a service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:42.292822 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 991/1962] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:42.295657 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c834f7b-389d-45be-a886-2b03fdc9be97 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:42.296257 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c834f7b-389d-45be-a886-2b03fdc9be97 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:42.296802 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c834f7b-389d-45be-a886-2b03fdc9be97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:42.321622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-4c834f7b-389d-45be-a886-2b03fdc9be97 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:42.323275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4c834f7b-389d-45be-a886-2b03fdc9be97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:42.356807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff0dd5-9172-46ba-9317-a032e2c510d5 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:42.364017 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803c7da-2bb2-4b78-88e7-b89da886ac8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:42.366390 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803c7da-2bb2-4b78-88e7-b89da886ac8f service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:42.366390 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803c7da-2bb2-4b78-88e7-b89da886ac8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:42.366629 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803c7da-2bb2-4b78-88e7-b89da886ac8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:42.372109 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c803c7da-2bb2-4b78-88e7-b89da886ac8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:42.451897 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 992/1963] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:42.472135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:42.472508 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c834f7b-389d-45be-a886-2b03fdc9be97 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:42.489692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c834f7b-389d-45be-a886-2b03fdc9be97 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:42.498113 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803c7da-2bb2-4b78-88e7-b89da886ac8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:42.504918 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803c7da-2bb2-4b78-88e7-b89da886ac8f service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:42.505124 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f46a9-109c-4944-9fab-4c04d02445bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:42.505124 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f46a9-109c-4944-9fab-4c04d02445bd service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:42.505124 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f46a9-109c-4944-9fab-4c04d02445bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:42.505124 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f46a9-109c-4944-9fab-4c04d02445bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:42.516201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 972/1964] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:42.531927 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f46a9-109c-4944-9fab-4c04d02445bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:42.533454 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f46a9-109c-4944-9fab-4c04d02445bd service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:42.549814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 993/1965] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:39:42.936604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803c7da-2bb2-4b78-88e7-b89da886ac8f service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:42.939248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c803c7da-2bb2-4b78-88e7-b89da886ac8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:43.000293 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803c7da-2bb2-4b78-88e7-b89da886ac8f service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:43.015084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfce2e7-bd66-4264-be7d-cb8684b39df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:43.016015 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfce2e7-bd66-4264-be7d-cb8684b39df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:43.016308 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfce2e7-bd66-4264-be7d-cb8684b39df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:43.016611 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfce2e7-bd66-4264-be7d-cb8684b39df3 admin admin] PATH_INFO: `/v3/projects/5c82bebd392e41348bfb17685328d8c4` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:43.018059 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfce2e7-bd66-4264-be7d-cb8684b39df3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c82bebd392e41348bfb17685328d8c4)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:43.020050 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfce2e7-bd66-4264-be7d-cb8684b39df3 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:43.154853 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-3cfce2e7-bd66-4264-be7d-cb8684b39df3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c82bebd392e41348bfb17685328d8c4', 'request_id': 'req-3cfce2e7-bd66-4264-be7d-cb8684b39df3'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:43.168340 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-3cfce2e7-bd66-4264-be7d-cb8684b39df3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c82bebd392e41348bfb17685328d8c4', 'request_id': 'req-3cfce2e7-bd66-4264-be7d-cb8684b39df3'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:43.183455 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-3cfce2e7-bd66-4264-be7d-cb8684b39df3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c82bebd392e41348bfb17685328d8c4', 'request_id': 'req-3cfce2e7-bd66-4264-be7d-cb8684b39df3'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:43.188634 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f46a9-109c-4944-9fab-4c04d02445bd service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:43.190336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2d2f46a9-109c-4944-9fab-4c04d02445bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:43.215311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f46a9-109c-4944-9fab-4c04d02445bd service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:43.217821 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-3cfce2e7-bd66-4264-be7d-cb8684b39df3 admin admin] The token cache is being invalidate because project 5c82bebd392e41348bfb17685328d8c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:43.218118 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-3cfce2e7-bd66-4264-be7d-cb8684b39df3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:43.218788 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-3cfce2e7-bd66-4264-be7d-cb8684b39df3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:43.220692 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 973/1966] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:42 2022] DELETE /identity/v3/projects/5c82bebd392e41348bfb17685328d8c4 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:43.225271 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec48c2-4c82-4f9a-8612-3188d8f3cde0 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:43.225792 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec48c2-4c82-4f9a-8612-3188d8f3cde0 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:43.226154 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec48c2-4c82-4f9a-8612-3188d8f3cde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:43.265641 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-46ec48c2-4c82-4f9a-8612-3188d8f3cde0 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:43.268167 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:43.269616 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-46ec48c2-4c82-4f9a-8612-3188d8f3cde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:43.293161 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f63874-e741-4532-8372-631b483fa8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:43.295549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f63874-e741-4532-8372-631b483fa8b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:43.295830 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f63874-e741-4532-8372-631b483fa8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:43.296105 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f63874-e741-4532-8372-631b483fa8b3 admin admin] PATH_INFO: `/v3/users/c82326fa82084c45a9a33eb0b4cc4375` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:43.297961 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f63874-e741-4532-8372-631b483fa8b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c82326fa82084c45a9a33eb0b4cc4375)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:43.300530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f63874-e741-4532-8372-631b483fa8b3 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:43.374240 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-84f63874-e741-4532-8372-631b483fa8b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c82326fa82084c45a9a33eb0b4cc4375'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:43.379119 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 974/1967] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:43.379841 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec48c2-4c82-4f9a-8612-3188d8f3cde0 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:43.381506 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-46ec48c2-4c82-4f9a-8612-3188d8f3cde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:43.386063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-84f63874-e741-4532-8372-631b483fa8b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c82326fa82084c45a9a33eb0b4cc4375'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:43.388109 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-84f63874-e741-4532-8372-631b483fa8b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c82326fa82084c45a9a33eb0b4cc4375'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:43.397160 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec48c2-4c82-4f9a-8612-3188d8f3cde0 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:43.403459 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619a6dc-13d0-41ff-b2a2-0bdf1a760fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:43.404444 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619a6dc-13d0-41ff-b2a2-0bdf1a760fcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:43.404800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619a6dc-13d0-41ff-b2a2-0bdf1a760fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:43.405227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619a6dc-13d0-41ff-b2a2-0bdf1a760fcc admin admin] PATH_INFO: `/v3/projects/f30ffab061894599b9e22de09a9c6623` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:43.406816 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a619a6dc-13d0-41ff-b2a2-0bdf1a760fcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f30ffab061894599b9e22de09a9c6623)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:43.408899 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a619a6dc-13d0-41ff-b2a2-0bdf1a760fcc admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:43.423924 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-84f63874-e741-4532-8372-631b483fa8b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c82326fa82084c45a9a33eb0b4cc4375'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:43.425092 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-84f63874-e741-4532-8372-631b483fa8b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c82326fa82084c45a9a33eb0b4cc4375'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:43.444957 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 994/1968] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:43 2022] DELETE /identity/v3/users/c82326fa82084c45a9a33eb0b4cc4375 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:39:43.446512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f7831-1d0d-4273-820e-de33ec739994 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:43.446797 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f7831-1d0d-4273-820e-de33ec739994 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:43.447136 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f7831-1d0d-4273-820e-de33ec739994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:43.473970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-141f7831-1d0d-4273-820e-de33ec739994 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:43.475656 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-141f7831-1d0d-4273-820e-de33ec739994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:43.482136 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a619a6dc-13d0-41ff-b2a2-0bdf1a760fcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f30ffab061894599b9e22de09a9c6623', 'request_id': 'req-a619a6dc-13d0-41ff-b2a2-0bdf1a760fcc'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:43.494395 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a619a6dc-13d0-41ff-b2a2-0bdf1a760fcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f30ffab061894599b9e22de09a9c6623', 'request_id': 'req-a619a6dc-13d0-41ff-b2a2-0bdf1a760fcc'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:43.504126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a619a6dc-13d0-41ff-b2a2-0bdf1a760fcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f30ffab061894599b9e22de09a9c6623', 'request_id': 'req-a619a6dc-13d0-41ff-b2a2-0bdf1a760fcc'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:43.506321 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:43.535501 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a619a6dc-13d0-41ff-b2a2-0bdf1a760fcc admin admin] The token cache is being invalidate because project f30ffab061894599b9e22de09a9c6623 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:43.536364 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a619a6dc-13d0-41ff-b2a2-0bdf1a760fcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:43.537146 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a619a6dc-13d0-41ff-b2a2-0bdf1a760fcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:43.539422 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 975/1969] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:43 2022] DELETE /identity/v3/projects/f30ffab061894599b9e22de09a9c6623 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:43.541528 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57841988-94eb-4066-8401-20a37af1067b None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:43.542156 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57841988-94eb-4066-8401-20a37af1067b None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:43.542710 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57841988-94eb-4066-8401-20a37af1067b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:43.571220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-57841988-94eb-4066-8401-20a37af1067b None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:43.572727 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-57841988-94eb-4066-8401-20a37af1067b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:43.585600 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 995/1970] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:43.588793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f7831-1d0d-4273-820e-de33ec739994 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:43.588793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-141f7831-1d0d-4273-820e-de33ec739994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:43.618690 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f7831-1d0d-4273-820e-de33ec739994 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:43.632993 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439bc70f-44f3-4855-8101-cce687e1df7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:43.634044 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bc70f-44f3-4855-8101-cce687e1df7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:43.634504 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bc70f-44f3-4855-8101-cce687e1df7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:43.635734 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bc70f-44f3-4855-8101-cce687e1df7a admin admin] PATH_INFO: `/v3/users/a1f4ce5ac5e64740bdb00f279fa16c84` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:43.639268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439bc70f-44f3-4855-8101-cce687e1df7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1f4ce5ac5e64740bdb00f279fa16c84)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:43.641715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439bc70f-44f3-4855-8101-cce687e1df7a admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:43.676900 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 976/1971] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:43.677828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57841988-94eb-4066-8401-20a37af1067b None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:43.690719 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57841988-94eb-4066-8401-20a37af1067b None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:43.706224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb13fe3-1199-4d18-a3c3-3dfcae4942fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:43.707397 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb13fe3-1199-4d18-a3c3-3dfcae4942fb admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:43.708575 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb13fe3-1199-4d18-a3c3-3dfcae4942fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:43.709358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb13fe3-1199-4d18-a3c3-3dfcae4942fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:43.711103 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb13fe3-1199-4d18-a3c3-3dfcae4942fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:39:43.711608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb13fe3-1199-4d18-a3c3-3dfcae4942fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:43.716483 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb13fe3-1199-4d18-a3c3-3dfcae4942fb admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:43.755000 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-439bc70f-44f3-4855-8101-cce687e1df7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1f4ce5ac5e64740bdb00f279fa16c84'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:43.757084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 977/1972] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:39:43.758727 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb13fe3-1199-4d18-a3c3-3dfcae4942fb admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:43.761271 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5bb13fe3-1199-4d18-a3c3-3dfcae4942fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:43.766732 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:43.769325 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-439bc70f-44f3-4855-8101-cce687e1df7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1f4ce5ac5e64740bdb00f279fa16c84'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:43.770288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-439bc70f-44f3-4855-8101-cce687e1df7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1f4ce5ac5e64740bdb00f279fa16c84'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:43.781135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb13fe3-1199-4d18-a3c3-3dfcae4942fb admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:43.788784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-439bc70f-44f3-4855-8101-cce687e1df7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1f4ce5ac5e64740bdb00f279fa16c84'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:43.789128 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf1369-1fe9-47a0-b26b-5b420f766398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:43.789774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-439bc70f-44f3-4855-8101-cce687e1df7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1f4ce5ac5e64740bdb00f279fa16c84'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:43.790213 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf1369-1fe9-47a0-b26b-5b420f766398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:43.790617 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf1369-1fe9-47a0-b26b-5b420f766398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:43.790890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf1369-1fe9-47a0-b26b-5b420f766398 admin admin] PATH_INFO: `/v3/projects/a002b16665154ebc8128e8c25dde575a` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:43.792427 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf1369-1fe9-47a0-b26b-5b420f766398 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a002b16665154ebc8128e8c25dde575a)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:43.795466 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf1369-1fe9-47a0-b26b-5b420f766398 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:43.817200 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 996/1973] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:43 2022] DELETE /identity/v3/users/a1f4ce5ac5e64740bdb00f279fa16c84 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:43.818280 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439bc70f-44f3-4855-8101-cce687e1df7a admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:43.820028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-439bc70f-44f3-4855-8101-cce687e1df7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:43.896082 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-34bf1369-1fe9-47a0-b26b-5b420f766398 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a002b16665154ebc8128e8c25dde575a', 'request_id': 'req-34bf1369-1fe9-47a0-b26b-5b420f766398'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:43.906802 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-34bf1369-1fe9-47a0-b26b-5b420f766398 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a002b16665154ebc8128e8c25dde575a', 'request_id': 'req-34bf1369-1fe9-47a0-b26b-5b420f766398'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:43.920459 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-34bf1369-1fe9-47a0-b26b-5b420f766398 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a002b16665154ebc8128e8c25dde575a', 'request_id': 'req-34bf1369-1fe9-47a0-b26b-5b420f766398'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:43.942604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-34bf1369-1fe9-47a0-b26b-5b420f766398 admin admin] The token cache is being invalidate because project a002b16665154ebc8128e8c25dde575a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:43.943281 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-34bf1369-1fe9-47a0-b26b-5b420f766398 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:43.944084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-34bf1369-1fe9-47a0-b26b-5b420f766398 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:43.951415 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 978/1974] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:43 2022] DELETE /identity/v3/projects/a002b16665154ebc8128e8c25dde575a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:43.953652 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf1369-1fe9-47a0-b26b-5b420f766398 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:43.957062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-34bf1369-1fe9-47a0-b26b-5b420f766398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:43.980447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439bc70f-44f3-4855-8101-cce687e1df7a admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:44.086999 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf1369-1fe9-47a0-b26b-5b420f766398 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:44.106741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62ac1f-0ca9-48c9-9063-6a6b93574154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:44.108397 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62ac1f-0ca9-48c9-9063-6a6b93574154 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:44.109092 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62ac1f-0ca9-48c9-9063-6a6b93574154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:44.109714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62ac1f-0ca9-48c9-9063-6a6b93574154 admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:44.110786 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff62ac1f-0ca9-48c9-9063-6a6b93574154 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:44.114523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff62ac1f-0ca9-48c9-9063-6a6b93574154 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:44.128421 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e74b1c-e62c-4d2e-bf68-a0e954a6623a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:44.129454 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e74b1c-e62c-4d2e-bf68-a0e954a6623a service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:44.129828 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e74b1c-e62c-4d2e-bf68-a0e954a6623a service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:44.130171 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e74b1c-e62c-4d2e-bf68-a0e954a6623a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:44.132244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:44.132393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-68e74b1c-e62c-4d2e-bf68-a0e954a6623a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:44.161881 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e74b1c-e62c-4d2e-bf68-a0e954a6623a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:44.164496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e74b1c-e62c-4d2e-bf68-a0e954a6623a service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:44.177312 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 979/1975] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:43 2022] POST /identity/v3/projects => generated 375 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:44.177947 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62ac1f-0ca9-48c9-9063-6a6b93574154 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:44.179029 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 997/1976] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:44.181303 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e74b1c-e62c-4d2e-bf68-a0e954a6623a service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:44.184318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-ff62ac1f-0ca9-48c9-9063-6a6b93574154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:44.197223 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e74b1c-e62c-4d2e-bf68-a0e954a6623a service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:44.329196 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62ac1f-0ca9-48c9-9063-6a6b93574154 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:44.335936 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa66f78b-9ae2-45fe-894a-5cf366009ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:44.336086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c21997-b4e8-4983-9619-14158ddfb361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:44.336202 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66f78b-9ae2-45fe-894a-5cf366009ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:44.336202 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66f78b-9ae2-45fe-894a-5cf366009ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:44.336202 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66f78b-9ae2-45fe-894a-5cf366009ca6 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:44.336548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c21997-b4e8-4983-9619-14158ddfb361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:44.336548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c21997-b4e8-4983-9619-14158ddfb361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:44.336548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c21997-b4e8-4983-9619-14158ddfb361 admin admin] PATH_INFO: `/v3/users/a11ba8dee55b4457a043baf7b91dfa9e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:44.336732 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa66f78b-9ae2-45fe-894a-5cf366009ca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:44.337723 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa66f78b-9ae2-45fe-894a-5cf366009ca6 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:44.337922 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c21997-b4e8-4983-9619-14158ddfb361 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a11ba8dee55b4457a043baf7b91dfa9e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:44.339832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c21997-b4e8-4983-9619-14158ddfb361 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:44.378227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 998/1977] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:44 2022] POST /identity/v3/users => generated 288 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 04:39:44.382692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa66f78b-9ae2-45fe-894a-5cf366009ca6 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:44.384530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-aa66f78b-9ae2-45fe-894a-5cf366009ca6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:44.399288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa66f78b-9ae2-45fe-894a-5cf366009ca6 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:44.405699 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5cd434-b9a9-4134-96df-b14bc0fad838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:44.407519 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cd434-b9a9-4134-96df-b14bc0fad838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:44.407519 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cd434-b9a9-4134-96df-b14bc0fad838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:44.407731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cd434-b9a9-4134-96df-b14bc0fad838 admin admin] PATH_INFO: `/v3/projects/30081bed396a41dda6e1f9c8d1ed09ed` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:44.409439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5cd434-b9a9-4134-96df-b14bc0fad838 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30081bed396a41dda6e1f9c8d1ed09ed)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:44.412152 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5cd434-b9a9-4134-96df-b14bc0fad838 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:44.430973 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-16c21997-b4e8-4983-9619-14158ddfb361 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a11ba8dee55b4457a043baf7b91dfa9e'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:44.432288 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-16c21997-b4e8-4983-9619-14158ddfb361 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a11ba8dee55b4457a043baf7b91dfa9e'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:44.444778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:44.452448 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-16c21997-b4e8-4983-9619-14158ddfb361 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a11ba8dee55b4457a043baf7b91dfa9e'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:44.466416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-16c21997-b4e8-4983-9619-14158ddfb361 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a11ba8dee55b4457a043baf7b91dfa9e'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:44.468430 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-16c21997-b4e8-4983-9619-14158ddfb361 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a11ba8dee55b4457a043baf7b91dfa9e'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:44.487710 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ca5cd434-b9a9-4134-96df-b14bc0fad838 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30081bed396a41dda6e1f9c8d1ed09ed', 'request_id': 'req-ca5cd434-b9a9-4134-96df-b14bc0fad838'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:44.492538 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 980/1978] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:44 2022] DELETE /identity/v3/users/a11ba8dee55b4457a043baf7b91dfa9e => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:44.493708 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c21997-b4e8-4983-9619-14158ddfb361 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:44.495137 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-16c21997-b4e8-4983-9619-14158ddfb361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:44.518120 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ca5cd434-b9a9-4134-96df-b14bc0fad838 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30081bed396a41dda6e1f9c8d1ed09ed', 'request_id': 'req-ca5cd434-b9a9-4134-96df-b14bc0fad838'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:44.527729 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ca5cd434-b9a9-4134-96df-b14bc0fad838 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30081bed396a41dda6e1f9c8d1ed09ed', 'request_id': 'req-ca5cd434-b9a9-4134-96df-b14bc0fad838'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:44.545524 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ca5cd434-b9a9-4134-96df-b14bc0fad838 admin admin] The token cache is being invalidate because project 30081bed396a41dda6e1f9c8d1ed09ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:44.546035 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ca5cd434-b9a9-4134-96df-b14bc0fad838 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:44.547003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ca5cd434-b9a9-4134-96df-b14bc0fad838 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:44.549625 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 999/1979] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:44 2022] DELETE /identity/v3/projects/30081bed396a41dda6e1f9c8d1ed09ed => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:44.554393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02309fe6-d7b0-4955-a0b2-0c57ea271fc2 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:44.554972 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02309fe6-d7b0-4955-a0b2-0c57ea271fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:44.555434 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02309fe6-d7b0-4955-a0b2-0c57ea271fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:44.581357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-02309fe6-d7b0-4955-a0b2-0c57ea271fc2 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:44.582905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-02309fe6-d7b0-4955-a0b2-0c57ea271fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:44.630001 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c21997-b4e8-4983-9619-14158ddfb361 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:44.643444 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93273a19-fa4d-47bd-b509-487b2758fca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:44.645066 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93273a19-fa4d-47bd-b509-487b2758fca6 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:44.646320 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93273a19-fa4d-47bd-b509-487b2758fca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:44.647070 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93273a19-fa4d-47bd-b509-487b2758fca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:44.648034 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93273a19-fa4d-47bd-b509-487b2758fca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:44.652358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93273a19-fa4d-47bd-b509-487b2758fca6 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:44.686869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 981/1980] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:44.692424 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93273a19-fa4d-47bd-b509-487b2758fca6 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:44.694492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-93273a19-fa4d-47bd-b509-487b2758fca6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:44.726966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:44.737639 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93273a19-fa4d-47bd-b509-487b2758fca6 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:44.752514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6bfe2-e30d-4bf5-bba3-b7d75ca415cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:44.753440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6bfe2-e30d-4bf5-bba3-b7d75ca415cf admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:44.753945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6bfe2-e30d-4bf5-bba3-b7d75ca415cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:44.754374 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6bfe2-e30d-4bf5-bba3-b7d75ca415cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:44.755262 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f6bfe2-e30d-4bf5-bba3-b7d75ca415cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:44.757114 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f6bfe2-e30d-4bf5-bba3-b7d75ca415cf admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:44.757608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1000/1981] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:39:44.774790 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 982/1982] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:44.778983 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8276be-5e20-45cd-aeeb-b0820f43e3e5 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:44.779327 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8276be-5e20-45cd-aeeb-b0820f43e3e5 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:44.779560 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8276be-5e20-45cd-aeeb-b0820f43e3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:44.781400 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6bfe2-e30d-4bf5-bba3-b7d75ca415cf admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:44.795069 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6bfe2-e30d-4bf5-bba3-b7d75ca415cf admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:44.801111 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97efd4-b5f8-42eb-a7c7-50e1dd802ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:44.802410 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97efd4-b5f8-42eb-a7c7-50e1dd802ef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:44.802924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97efd4-b5f8-42eb-a7c7-50e1dd802ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:44.803342 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97efd4-b5f8-42eb-a7c7-50e1dd802ef0 admin admin] PATH_INFO: `/v3/projects/4f04dacfa01e42539204572c99bed3f6/users/13679676132642b88b2c88b58c4afc55/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:44.819429 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-aa8276be-5e20-45cd-aeeb-b0820f43e3e5 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:44.821123 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-aa8276be-5e20-45cd-aeeb-b0820f43e3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:44.824757 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef97efd4-b5f8-42eb-a7c7-50e1dd802ef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f04dacfa01e42539204572c99bed3f6, user_id=13679676132642b88b2c88b58c4afc55, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:44.827979 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef97efd4-b5f8-42eb-a7c7-50e1dd802ef0 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:44.847247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1001/1983] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:39:44.849413 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 983/1984] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:44 2022] PUT /identity/v3/projects/4f04dacfa01e42539204572c99bed3f6/users/13679676132642b88b2c88b58c4afc55/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:39:44.859139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8276be-5e20-45cd-aeeb-b0820f43e3e5 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:44.865873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911ca18-e075-4445-be33-42580c307afc None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:44.869567 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911ca18-e075-4445-be33-42580c307afc None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:44.870542 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911ca18-e075-4445-be33-42580c307afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:44.873236 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8276be-5e20-45cd-aeeb-b0820f43e3e5 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:44.943905 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-3911ca18-e075-4445-be33-42580c307afc None None] MFA Rules not processed for user `13679676132642b88b2c88b58c4afc55`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:44.947895 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-3911ca18-e075-4445-be33-42580c307afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:44.968957 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f78cd-3c03-4946-a437-3cc09e308bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:44.969627 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f78cd-3c03-4946-a437-3cc09e308bbd admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:44.969909 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f78cd-3c03-4946-a437-3cc09e308bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:44.970229 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f78cd-3c03-4946-a437-3cc09e308bbd admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:44.971434 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f78cd-3c03-4946-a437-3cc09e308bbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:39:44.971955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f78cd-3c03-4946-a437-3cc09e308bbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:44.974957 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f78cd-3c03-4946-a437-3cc09e308bbd admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:44.995638 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1002/1985] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:39:44.998131 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f78cd-3c03-4946-a437-3cc09e308bbd admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:45.010284 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f78cd-3c03-4946-a437-3cc09e308bbd admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:45.021059 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f030f-86de-474c-ae6c-18d1bd76debd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:45.022247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f030f-86de-474c-ae6c-18d1bd76debd admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:45.022851 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f030f-86de-474c-ae6c-18d1bd76debd admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:45.023368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f030f-86de-474c-ae6c-18d1bd76debd admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:45.024928 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f030f-86de-474c-ae6c-18d1bd76debd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:45.029345 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f030f-86de-474c-ae6c-18d1bd76debd admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:45.052704 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 984/1986] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:39:45.066166 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911ca18-e075-4445-be33-42580c307afc None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:45.068348 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-3911ca18-e075-4445-be33-42580c307afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:45.082579 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1003/1987] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:44 2022] POST /identity/v3/projects => generated 389 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:45.086677 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f030f-86de-474c-ae6c-18d1bd76debd admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:45.099381 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:45.190280 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911ca18-e075-4445-be33-42580c307afc None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:45.226392 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f030f-86de-474c-ae6c-18d1bd76debd admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:45.236699 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfea021-b6d7-4548-bb3e-69ac718c9422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:45.237226 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfea021-b6d7-4548-bb3e-69ac718c9422 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:45.237522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfea021-b6d7-4548-bb3e-69ac718c9422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:45.237832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfea021-b6d7-4548-bb3e-69ac718c9422 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:45.238593 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfea021-b6d7-4548-bb3e-69ac718c9422 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:45.240789 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfea021-b6d7-4548-bb3e-69ac718c9422 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:45.282158 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1004/1988] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:45 2022] POST /identity/v3/users => generated 295 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 04:39:45.286978 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfea021-b6d7-4548-bb3e-69ac718c9422 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:45.287126 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-adfea021-b6d7-4548-bb3e-69ac718c9422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:45.310767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfea021-b6d7-4548-bb3e-69ac718c9422 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:45.316837 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e260674-5d22-4657-94f0-cdfa9ee59824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:45.317513 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e260674-5d22-4657-94f0-cdfa9ee59824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:45.317848 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e260674-5d22-4657-94f0-cdfa9ee59824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:45.318189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e260674-5d22-4657-94f0-cdfa9ee59824 admin admin] PATH_INFO: `/v3/projects/fbe08d3108834be685db29cd44bbf4f1` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:45.320231 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e260674-5d22-4657-94f0-cdfa9ee59824 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbe08d3108834be685db29cd44bbf4f1)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:45.323823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e260674-5d22-4657-94f0-cdfa9ee59824 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:45.382405 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f6d13-eb38-4595-a077-039e820d514d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:45.383063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f6d13-eb38-4595-a077-039e820d514d service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:45.384009 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f6d13-eb38-4595-a077-039e820d514d service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:45.384009 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f6d13-eb38-4595-a077-039e820d514d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:45.443542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6e260674-5d22-4657-94f0-cdfa9ee59824 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbe08d3108834be685db29cd44bbf4f1', 'request_id': 'req-6e260674-5d22-4657-94f0-cdfa9ee59824'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:45.461856 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6e260674-5d22-4657-94f0-cdfa9ee59824 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbe08d3108834be685db29cd44bbf4f1', 'request_id': 'req-6e260674-5d22-4657-94f0-cdfa9ee59824'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:45.472621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:45.475483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6e260674-5d22-4657-94f0-cdfa9ee59824 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbe08d3108834be685db29cd44bbf4f1', 'request_id': 'req-6e260674-5d22-4657-94f0-cdfa9ee59824'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:45.518717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6e260674-5d22-4657-94f0-cdfa9ee59824 admin admin] The token cache is being invalidate because project fbe08d3108834be685db29cd44bbf4f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:45.519070 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6e260674-5d22-4657-94f0-cdfa9ee59824 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:45.520132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6e260674-5d22-4657-94f0-cdfa9ee59824 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:45.522659 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1005/1989] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:45 2022] DELETE /identity/v3/projects/fbe08d3108834be685db29cd44bbf4f1 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:45.523622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e260674-5d22-4657-94f0-cdfa9ee59824 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:45.526529 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6e260674-5d22-4657-94f0-cdfa9ee59824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:45.562503 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f6d13-eb38-4595-a077-039e820d514d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:45.564114 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f6d13-eb38-4595-a077-039e820d514d service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:45.585438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 985/1990] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:45.586963 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f6d13-eb38-4595-a077-039e820d514d service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:45.588307 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-f86f6d13-eb38-4595-a077-039e820d514d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:45.624175 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f6d13-eb38-4595-a077-039e820d514d service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:45.637615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b59db-1d1a-4829-bae3-21bdde5c2bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:45.638412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b59db-1d1a-4829-bae3-21bdde5c2bed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:45.638787 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b59db-1d1a-4829-bae3-21bdde5c2bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:45.639154 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b59db-1d1a-4829-bae3-21bdde5c2bed admin admin] PATH_INFO: `/v3/projects/86d7ea8d671b49b0b33b3539474649e8` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:45.643532 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b59db-1d1a-4829-bae3-21bdde5c2bed admin admin] RBAC: Authorizing `identity:delete_project(project_id=86d7ea8d671b49b0b33b3539474649e8)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:45.644253 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e260674-5d22-4657-94f0-cdfa9ee59824 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:45.645738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b59db-1d1a-4829-bae3-21bdde5c2bed admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:45.660984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9d864-53d8-4143-90bf-318d00bd2fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:45.662063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9d864-53d8-4143-90bf-318d00bd2fea admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:45.662457 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9d864-53d8-4143-90bf-318d00bd2fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:45.666860 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9d864-53d8-4143-90bf-318d00bd2fea admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:45.667963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c9d864-53d8-4143-90bf-318d00bd2fea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:45.675869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c9d864-53d8-4143-90bf-318d00bd2fea admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:45.735381 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1006/1991] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:45.736392 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9d864-53d8-4143-90bf-318d00bd2fea admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:45.745409 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b6c9d864-53d8-4143-90bf-318d00bd2fea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:45.758897 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-fb7b59db-1d1a-4829-bae3-21bdde5c2bed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86d7ea8d671b49b0b33b3539474649e8', 'request_id': 'req-fb7b59db-1d1a-4829-bae3-21bdde5c2bed'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:45.779498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-fb7b59db-1d1a-4829-bae3-21bdde5c2bed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86d7ea8d671b49b0b33b3539474649e8', 'request_id': 'req-fb7b59db-1d1a-4829-bae3-21bdde5c2bed'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:45.791783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9d864-53d8-4143-90bf-318d00bd2fea admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:45.812497 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-fb7b59db-1d1a-4829-bae3-21bdde5c2bed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86d7ea8d671b49b0b33b3539474649e8', 'request_id': 'req-fb7b59db-1d1a-4829-bae3-21bdde5c2bed'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:45.831993 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:45.840518 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-fb7b59db-1d1a-4829-bae3-21bdde5c2bed admin admin] The token cache is being invalidate because project 86d7ea8d671b49b0b33b3539474649e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:45.841322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-fb7b59db-1d1a-4829-bae3-21bdde5c2bed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:45.843550 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-fb7b59db-1d1a-4829-bae3-21bdde5c2bed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:45.847515 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 986/1992] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:45 2022] DELETE /identity/v3/projects/86d7ea8d671b49b0b33b3539474649e8 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:45.849063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b59db-1d1a-4829-bae3-21bdde5c2bed admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:45.856563 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-fb7b59db-1d1a-4829-bae3-21bdde5c2bed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:45.974338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0a088-731d-4194-8d4f-7dba008073f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:45.975069 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0a088-731d-4194-8d4f-7dba008073f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:45.975945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0a088-731d-4194-8d4f-7dba008073f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:45.976247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0a088-731d-4194-8d4f-7dba008073f2 admin admin] PATH_INFO: `/v3/users/267ae42789094b3ca13b319a0e54e609` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:45.977824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0a088-731d-4194-8d4f-7dba008073f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=267ae42789094b3ca13b319a0e54e609)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:45.987736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0a088-731d-4194-8d4f-7dba008073f2 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:46.052858 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b59db-1d1a-4829-bae3-21bdde5c2bed admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:46.073333 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a588e2a-a449-4424-84c4-1134e5735cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:46.074337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a588e2a-a449-4424-84c4-1134e5735cbd admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:46.075065 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a588e2a-a449-4424-84c4-1134e5735cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:46.075701 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a588e2a-a449-4424-84c4-1134e5735cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:46.076602 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a588e2a-a449-4424-84c4-1134e5735cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:46.078527 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a588e2a-a449-4424-84c4-1134e5735cbd admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:46.103126 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b8b0a088-731d-4194-8d4f-7dba008073f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '267ae42789094b3ca13b319a0e54e609'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:46.109006 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 987/1993] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:46.110616 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211e4c5-9bab-4f27-88b6-02203e5f4065 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:46.110906 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211e4c5-9bab-4f27-88b6-02203e5f4065 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:46.111472 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211e4c5-9bab-4f27-88b6-02203e5f4065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:46.125707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b8b0a088-731d-4194-8d4f-7dba008073f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '267ae42789094b3ca13b319a0e54e609'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:46.127177 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b8b0a088-731d-4194-8d4f-7dba008073f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '267ae42789094b3ca13b319a0e54e609'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:46.155505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-4211e4c5-9bab-4f27-88b6-02203e5f4065 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:46.156741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-4211e4c5-9bab-4f27-88b6-02203e5f4065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:46.160560 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b8b0a088-731d-4194-8d4f-7dba008073f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '267ae42789094b3ca13b319a0e54e609'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:46.163588 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b8b0a088-731d-4194-8d4f-7dba008073f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '267ae42789094b3ca13b319a0e54e609'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:46.165886 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 988/1994] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:46.166824 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4211e4c5-9bab-4f27-88b6-02203e5f4065 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:46.168225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-4211e4c5-9bab-4f27-88b6-02203e5f4065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:46.181536 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4211e4c5-9bab-4f27-88b6-02203e5f4065 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:46.188271 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b8baa-d696-4a95-ba8b-ec85ccc0abbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:46.189017 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b8baa-d696-4a95-ba8b-ec85ccc0abbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:46.189302 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b8baa-d696-4a95-ba8b-ec85ccc0abbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:46.189604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b8baa-d696-4a95-ba8b-ec85ccc0abbe admin admin] PATH_INFO: `/v3/projects/2db11129a2384e79af22b323ad1f08f2/users/cda28cd3d2424868bf57002e388d28af/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:46.191133 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1007/1995] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:45 2022] DELETE /identity/v3/users/267ae42789094b3ca13b319a0e54e609 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:46.192677 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b80ce-c0f5-40c2-9ebf-d43094f7f23f None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:46.192973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b80ce-c0f5-40c2-9ebf-d43094f7f23f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:46.193256 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b80ce-c0f5-40c2-9ebf-d43094f7f23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:46.208133 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b8baa-d696-4a95-ba8b-ec85ccc0abbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=2db11129a2384e79af22b323ad1f08f2, user_id=cda28cd3d2424868bf57002e388d28af, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:46.210936 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b8baa-d696-4a95-ba8b-ec85ccc0abbe admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:46.221775 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-3e3b80ce-c0f5-40c2-9ebf-d43094f7f23f None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:46.223806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3e3b80ce-c0f5-40c2-9ebf-d43094f7f23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:46.255624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 989/1996] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:46 2022] PUT /identity/v3/projects/2db11129a2384e79af22b323ad1f08f2/users/cda28cd3d2424868bf57002e388d28af/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:46.270571 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4f632-a4c0-46b2-b9b8-61d23e625793 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:46.270903 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4f632-a4c0-46b2-b9b8-61d23e625793 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:46.271123 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4f632-a4c0-46b2-b9b8-61d23e625793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:46.347527 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-f2c4f632-a4c0-46b2-b9b8-61d23e625793 None None] MFA Rules not processed for user `cda28cd3d2424868bf57002e388d28af`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:46.352523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-f2c4f632-a4c0-46b2-b9b8-61d23e625793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:46.369773 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:46.379383 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1008/1997] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:46.381029 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b80ce-c0f5-40c2-9ebf-d43094f7f23f None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:46.394512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b80ce-c0f5-40c2-9ebf-d43094f7f23f None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:46.450326 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:46.475091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 990/1998] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:46.476215 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4f632-a4c0-46b2-b9b8-61d23e625793 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:46.490293 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae717c-03f7-4bd8-84ca-799dd3041724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:46.490990 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae717c-03f7-4bd8-84ca-799dd3041724 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:46.491639 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae717c-03f7-4bd8-84ca-799dd3041724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:46.492116 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae717c-03f7-4bd8-84ca-799dd3041724 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:46.493269 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae717c-03f7-4bd8-84ca-799dd3041724 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:39:46.493719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae717c-03f7-4bd8-84ca-799dd3041724 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:46.497393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae717c-03f7-4bd8-84ca-799dd3041724 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:46.529197 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1009/1999] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:39:46.530511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae717c-03f7-4bd8-84ca-799dd3041724 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:46.546229 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae717c-03f7-4bd8-84ca-799dd3041724 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:46.553980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105eb832-8e15-4297-85bd-425ded0ad063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:46.555121 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105eb832-8e15-4297-85bd-425ded0ad063 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:46.555509 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105eb832-8e15-4297-85bd-425ded0ad063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:46.555884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105eb832-8e15-4297-85bd-425ded0ad063 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:46.556002 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4f632-a4c0-46b2-b9b8-61d23e625793 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:46.556782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105eb832-8e15-4297-85bd-425ded0ad063 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:46.560547 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105eb832-8e15-4297-85bd-425ded0ad063 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:46.562484 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e42398-0fa5-423f-a834-3db4f00ba62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:46.563195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e42398-0fa5-423f-a834-3db4f00ba62f admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:46.563596 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e42398-0fa5-423f-a834-3db4f00ba62f admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:46.563938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e42398-0fa5-423f-a834-3db4f00ba62f admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:46.564787 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e42398-0fa5-423f-a834-3db4f00ba62f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:46.567742 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e42398-0fa5-423f-a834-3db4f00ba62f admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:46.626657 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1010/2000] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:46 2022] POST /identity/v3/projects => generated 409 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:46.631970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105eb832-8e15-4297-85bd-425ded0ad063 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:46.636117 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 991/2001] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:46 2022] POST /identity/v3/projects => generated 389 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:46.644004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e42398-0fa5-423f-a834-3db4f00ba62f admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:46.766332 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105eb832-8e15-4297-85bd-425ded0ad063 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:46.798889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e42398-0fa5-423f-a834-3db4f00ba62f admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:46.815010 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212cc9d-5b52-4eb5-84d9-b9b678a92070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:46.815787 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212cc9d-5b52-4eb5-84d9-b9b678a92070 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:46.816169 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212cc9d-5b52-4eb5-84d9-b9b678a92070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:46.816523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212cc9d-5b52-4eb5-84d9-b9b678a92070 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:46.817414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212cc9d-5b52-4eb5-84d9-b9b678a92070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:46.821285 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212cc9d-5b52-4eb5-84d9-b9b678a92070 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:46.837675 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a1ee3-3d85-4fb0-80fe-ed0d8014fa97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:46.839065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a1ee3-3d85-4fb0-80fe-ed0d8014fa97 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:46.839065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a1ee3-3d85-4fb0-80fe-ed0d8014fa97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:46.839065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a1ee3-3d85-4fb0-80fe-ed0d8014fa97 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:46.840495 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a1ee3-3d85-4fb0-80fe-ed0d8014fa97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:46.843559 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a1ee3-3d85-4fb0-80fe-ed0d8014fa97 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:46.868543 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 992/2002] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:46 2022] POST /identity/v3/users => generated 301 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:46.870037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212cc9d-5b52-4eb5-84d9-b9b678a92070 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:46.876828 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1011/2003] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:46 2022] POST /identity/v3/users => generated 305 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 04:39:46.880884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a1ee3-3d85-4fb0-80fe-ed0d8014fa97 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:46.889327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212cc9d-5b52-4eb5-84d9-b9b678a92070 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:46.896090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a1ee3-3d85-4fb0-80fe-ed0d8014fa97 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:46.900827 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a56dd1-8be8-43d0-80ae-7823856bf65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:46.901469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a56dd1-8be8-43d0-80ae-7823856bf65b admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:46.901775 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a56dd1-8be8-43d0-80ae-7823856bf65b admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:46.902072 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a56dd1-8be8-43d0-80ae-7823856bf65b admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:46.903337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a56dd1-8be8-43d0-80ae-7823856bf65b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:46.905043 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a56dd1-8be8-43d0-80ae-7823856bf65b admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:46.913804 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a213dc2-4042-40cd-aed2-dbe7328cdc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:46.914344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a213dc2-4042-40cd-aed2-dbe7328cdc0d admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:46.914886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a213dc2-4042-40cd-aed2-dbe7328cdc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:46.914886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a213dc2-4042-40cd-aed2-dbe7328cdc0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:46.915599 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a213dc2-4042-40cd-aed2-dbe7328cdc0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:46.917243 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a213dc2-4042-40cd-aed2-dbe7328cdc0d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:46.929687 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 993/2004] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:46.937659 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1012/2005] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 04:39:46.937827 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a56dd1-8be8-43d0-80ae-7823856bf65b admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:46.942521 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a213dc2-4042-40cd-aed2-dbe7328cdc0d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:46.954479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a56dd1-8be8-43d0-80ae-7823856bf65b admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:46.960894 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a213dc2-4042-40cd-aed2-dbe7328cdc0d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:46.964117 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69a522-b577-41b7-b9f7-e4a96290237d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:46.964816 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69a522-b577-41b7-b9f7-e4a96290237d admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:46.965680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69a522-b577-41b7-b9f7-e4a96290237d admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:46.965942 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69a522-b577-41b7-b9f7-e4a96290237d admin admin] PATH_INFO: `/v3/projects/b6e32d2a6b3344ddb02a6038ced90c66/users/09ad26801c6f49f29e5ca2e49dc935fc/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:46.976883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:46.979120 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb0652-631c-4e52-b1c6-ee9c5c3f6131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:46.979773 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb0652-631c-4e52-b1c6-ee9c5c3f6131 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:46.980079 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb0652-631c-4e52-b1c6-ee9c5c3f6131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:46.980334 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb0652-631c-4e52-b1c6-ee9c5c3f6131 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:46.982034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb0652-631c-4e52-b1c6-ee9c5c3f6131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:46.983473 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb0652-631c-4e52-b1c6-ee9c5c3f6131 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:47.001603 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1013/2006] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:47.001804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d69a522-b577-41b7-b9f7-e4a96290237d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6e32d2a6b3344ddb02a6038ced90c66, user_id=09ad26801c6f49f29e5ca2e49dc935fc, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:47.004157 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb0652-631c-4e52-b1c6-ee9c5c3f6131 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:47.005613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d69a522-b577-41b7-b9f7-e4a96290237d admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:47.027063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb0652-631c-4e52-b1c6-ee9c5c3f6131 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:47.041013 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 994/2007] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:46 2022] PUT /identity/v3/projects/b6e32d2a6b3344ddb02a6038ced90c66/users/09ad26801c6f49f29e5ca2e49dc935fc/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:47.046316 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69a522-b577-41b7-b9f7-e4a96290237d admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:47.084890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:47.178490 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09570a-da88-4b1f-8977-d4a7f3ddb859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:47.179187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09570a-da88-4b1f-8977-d4a7f3ddb859 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:47.179476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09570a-da88-4b1f-8977-d4a7f3ddb859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:47.179750 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09570a-da88-4b1f-8977-d4a7f3ddb859 admin admin] PATH_INFO: `/v3/projects/e2032f1adcd94de18d309bce9becf7eb/users/a9c89324e7d3449e9d805105e657ccab/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:47.194593 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69a522-b577-41b7-b9f7-e4a96290237d admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:47.205334 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6483f-dc2d-4f04-89dd-84ee2aa38ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:47.205650 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6483f-dc2d-4f04-89dd-84ee2aa38ecd admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:47.206008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6483f-dc2d-4f04-89dd-84ee2aa38ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:47.206229 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6483f-dc2d-4f04-89dd-84ee2aa38ecd admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:47.206876 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6483f-dc2d-4f04-89dd-84ee2aa38ecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:47.211001 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6483f-dc2d-4f04-89dd-84ee2aa38ecd admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:47.214173 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f09570a-da88-4b1f-8977-d4a7f3ddb859 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2032f1adcd94de18d309bce9becf7eb, user_id=a9c89324e7d3449e9d805105e657ccab, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:47.218545 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f09570a-da88-4b1f-8977-d4a7f3ddb859 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:47.231440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 995/2008] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:47.232490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6483f-dc2d-4f04-89dd-84ee2aa38ecd admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:47.236623 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-8bf6483f-dc2d-4f04-89dd-84ee2aa38ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:47.253452 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1014/2009] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:47 2022] PUT /identity/v3/projects/e2032f1adcd94de18d309bce9becf7eb/users/a9c89324e7d3449e9d805105e657ccab/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:47.253452 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09570a-da88-4b1f-8977-d4a7f3ddb859 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:47.259412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6483f-dc2d-4f04-89dd-84ee2aa38ecd admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:47.391246 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09570a-da88-4b1f-8977-d4a7f3ddb859 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:47.397428 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f137a7-80f5-4f9a-9a04-461389376795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:47.398206 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f137a7-80f5-4f9a-9a04-461389376795 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:47.398494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f137a7-80f5-4f9a-9a04-461389376795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:47.398776 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f137a7-80f5-4f9a-9a04-461389376795 admin admin] PATH_INFO: `/v3/projects/b6e32d2a6b3344ddb02a6038ced90c66/users/09ad26801c6f49f29e5ca2e49dc935fc/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:47.401231 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f137a7-80f5-4f9a-9a04-461389376795 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6e32d2a6b3344ddb02a6038ced90c66, user_id=09ad26801c6f49f29e5ca2e49dc935fc, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:47.404698 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f137a7-80f5-4f9a-9a04-461389376795 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:47.417465 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1015/2010] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:47 2022] PUT /identity/v3/projects/b6e32d2a6b3344ddb02a6038ced90c66/users/09ad26801c6f49f29e5ca2e49dc935fc/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:47.419893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be9004-6635-479a-8c3a-f5fd3fa4c4d1 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:47.420297 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be9004-6635-479a-8c3a-f5fd3fa4c4d1 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:47.420580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be9004-6635-479a-8c3a-f5fd3fa4c4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:47.438734 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764879f0-47e1-4869-9df4-19321b704317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:47.440484 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764879f0-47e1-4869-9df4-19321b704317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:47.440767 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764879f0-47e1-4869-9df4-19321b704317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:47.441044 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764879f0-47e1-4869-9df4-19321b704317 admin admin] PATH_INFO: `/v3/projects/7f49e0e780c549f598f0770fdcf88d14` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:47.442382 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764879f0-47e1-4869-9df4-19321b704317 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f49e0e780c549f598f0770fdcf88d14)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:47.445098 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764879f0-47e1-4869-9df4-19321b704317 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:47.482558 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-54be9004-6635-479a-8c3a-f5fd3fa4c4d1 None None] MFA Rules not processed for user `a9c89324e7d3449e9d805105e657ccab`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:47.483805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-54be9004-6635-479a-8c3a-f5fd3fa4c4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:47.520704 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-764879f0-47e1-4869-9df4-19321b704317 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f49e0e780c549f598f0770fdcf88d14', 'request_id': 'req-764879f0-47e1-4869-9df4-19321b704317'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:47.543454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-764879f0-47e1-4869-9df4-19321b704317 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f49e0e780c549f598f0770fdcf88d14', 'request_id': 'req-764879f0-47e1-4869-9df4-19321b704317'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:47.565863 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-764879f0-47e1-4869-9df4-19321b704317 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f49e0e780c549f598f0770fdcf88d14', 'request_id': 'req-764879f0-47e1-4869-9df4-19321b704317'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:47.580273 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-764879f0-47e1-4869-9df4-19321b704317 admin admin] The token cache is being invalidate because project 7f49e0e780c549f598f0770fdcf88d14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:47.580571 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-764879f0-47e1-4869-9df4-19321b704317 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:47.581269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-764879f0-47e1-4869-9df4-19321b704317 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:47.583328 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 996/2011] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:47 2022] DELETE /identity/v3/projects/7f49e0e780c549f598f0770fdcf88d14 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:47.585034 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f46300-c464-4f97-ba90-e144e2fc19c0 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:47.585322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f46300-c464-4f97-ba90-e144e2fc19c0 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:47.585617 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f46300-c464-4f97-ba90-e144e2fc19c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:47.607383 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1016/2012] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:47 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:47.609086 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f45c2-7487-4a49-8e71-e7b5114cc851 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:47.609514 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f45c2-7487-4a49-8e71-e7b5114cc851 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:47.609945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f45c2-7487-4a49-8e71-e7b5114cc851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:47.632057 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:47.641156 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-a4f46300-c464-4f97-ba90-e144e2fc19c0 None None] MFA Rules not processed for user `09ad26801c6f49f29e5ca2e49dc935fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:47.644617 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-a4f46300-c464-4f97-ba90-e144e2fc19c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:47.696467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-179f45c2-7487-4a49-8e71-e7b5114cc851 None None] MFA Rules not processed for user `13679676132642b88b2c88b58c4afc55`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:47.697982 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-179f45c2-7487-4a49-8e71-e7b5114cc851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:47.758076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:47.811919 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1017/2013] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:47.812475 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f45c2-7487-4a49-8e71-e7b5114cc851 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:47.814190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-179f45c2-7487-4a49-8e71-e7b5114cc851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:47.851509 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 997/2014] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:47 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:47.854499 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f46300-c464-4f97-ba90-e144e2fc19c0 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:47.857112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-a4f46300-c464-4f97-ba90-e144e2fc19c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:47.971391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f45c2-7487-4a49-8e71-e7b5114cc851 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:47.977738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d73b6-6566-48f6-82e9-9244639e2465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:47.978470 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d73b6-6566-48f6-82e9-9244639e2465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:47.978750 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d73b6-6566-48f6-82e9-9244639e2465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:47.979026 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d73b6-6566-48f6-82e9-9244639e2465 admin admin] PATH_INFO: `/v3/users/e5d59349482d479c9450c106025850f6` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:47.980658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d73b6-6566-48f6-82e9-9244639e2465 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5d59349482d479c9450c106025850f6)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:47.984876 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d73b6-6566-48f6-82e9-9244639e2465 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:47.992742 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f46300-c464-4f97-ba90-e144e2fc19c0 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:47.999391 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba3538-df8b-46ea-8f63-09e6567971df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:48.000528 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba3538-df8b-46ea-8f63-09e6567971df admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:48.000958 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba3538-df8b-46ea-8f63-09e6567971df admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:48.001403 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba3538-df8b-46ea-8f63-09e6567971df admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:48.002249 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba3538-df8b-46ea-8f63-09e6567971df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:48.004324 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba3538-df8b-46ea-8f63-09e6567971df admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:48.040248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 998/2015] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:47 2022] POST /identity/v3/projects => generated 409 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:48.042064 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba3538-df8b-46ea-8f63-09e6567971df admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:48.044821 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-cbba3538-df8b-46ea-8f63-09e6567971df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:48.070976 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-306d73b6-6566-48f6-82e9-9244639e2465 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5d59349482d479c9450c106025850f6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:48.082503 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-306d73b6-6566-48f6-82e9-9244639e2465 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5d59349482d479c9450c106025850f6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:48.084054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-306d73b6-6566-48f6-82e9-9244639e2465 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5d59349482d479c9450c106025850f6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:48.106628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-306d73b6-6566-48f6-82e9-9244639e2465 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5d59349482d479c9450c106025850f6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:48.108453 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-306d73b6-6566-48f6-82e9-9244639e2465 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5d59349482d479c9450c106025850f6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:48.135432 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1018/2016] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:47 2022] DELETE /identity/v3/users/e5d59349482d479c9450c106025850f6 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:48.136436 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d73b6-6566-48f6-82e9-9244639e2465 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:48.138653 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-306d73b6-6566-48f6-82e9-9244639e2465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:48.142384 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba3538-df8b-46ea-8f63-09e6567971df admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:48.203305 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:48.217309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696b70e-48e9-4903-af67-78e2759d0f0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:48.219547 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696b70e-48e9-4903-af67-78e2759d0f0e service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:48.219889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696b70e-48e9-4903-af67-78e2759d0f0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:48.220510 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696b70e-48e9-4903-af67-78e2759d0f0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:48.223490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c696b70e-48e9-4903-af67-78e2759d0f0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:48.236370 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:48.246892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d73b6-6566-48f6-82e9-9244639e2465 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:48.256817 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678247b7-05cd-4afb-8d55-5a0e21fad0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:48.258068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678247b7-05cd-4afb-8d55-5a0e21fad0aa admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:48.258464 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678247b7-05cd-4afb-8d55-5a0e21fad0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:48.258819 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678247b7-05cd-4afb-8d55-5a0e21fad0aa admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:48.260313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678247b7-05cd-4afb-8d55-5a0e21fad0aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:48.269239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678247b7-05cd-4afb-8d55-5a0e21fad0aa admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:48.316298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1019/2017] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:48 2022] POST /identity/v3/users => generated 311 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:48.318622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678247b7-05cd-4afb-8d55-5a0e21fad0aa admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:48.331794 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c696b70e-48e9-4903-af67-78e2759d0f0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:48.334427 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c696b70e-48e9-4903-af67-78e2759d0f0e service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:48.343452 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678247b7-05cd-4afb-8d55-5a0e21fad0aa admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:48.347909 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 999/2018] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:48 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:48.359656 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c0c16-25d3-4fdf-baf7-6ec6a6d3fe5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:48.360727 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c0c16-25d3-4fdf-baf7-6ec6a6d3fe5e admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:48.361010 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c0c16-25d3-4fdf-baf7-6ec6a6d3fe5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:48.361472 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c0c16-25d3-4fdf-baf7-6ec6a6d3fe5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:48.362583 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69c0c16-25d3-4fdf-baf7-6ec6a6d3fe5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:48.364216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69c0c16-25d3-4fdf-baf7-6ec6a6d3fe5e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:48.376091 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1020/2019] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:48.388745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696b70e-48e9-4903-af67-78e2759d0f0e service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:48.403751 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696b70e-48e9-4903-af67-78e2759d0f0e service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:48.414873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced95c0-5e14-409e-a224-cdc436128c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:48.415789 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced95c0-5e14-409e-a224-cdc436128c21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:48.416018 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced95c0-5e14-409e-a224-cdc436128c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:48.416368 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced95c0-5e14-409e-a224-cdc436128c21 admin admin] PATH_INFO: `/v3/projects/b3efbf2bb946439d89f790737784f630/users/353cfbda09e34082bbed614067aea039/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:48.437801 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced95c0-5e14-409e-a224-cdc436128c21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3efbf2bb946439d89f790737784f630, user_id=353cfbda09e34082bbed614067aea039, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:48.441110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced95c0-5e14-409e-a224-cdc436128c21 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:48.469799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1000/2020] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:48 2022] PUT /identity/v3/projects/b3efbf2bb946439d89f790737784f630/users/353cfbda09e34082bbed614067aea039/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:48.473822 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c0c16-25d3-4fdf-baf7-6ec6a6d3fe5e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:48.489145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c0c16-25d3-4fdf-baf7-6ec6a6d3fe5e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:48.563043 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc4f6c-4d77-4d05-8370-c8b13f4069b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:48.563678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc4f6c-4d77-4d05-8370-c8b13f4069b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:48.564006 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc4f6c-4d77-4d05-8370-c8b13f4069b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:48.564279 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc4f6c-4d77-4d05-8370-c8b13f4069b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:48.565035 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc4f6c-4d77-4d05-8370-c8b13f4069b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:48.567588 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc4f6c-4d77-4d05-8370-c8b13f4069b9 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:48.577021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced95c0-5e14-409e-a224-cdc436128c21 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:48.579017 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-1ced95c0-5e14-409e-a224-cdc436128c21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:48.606779 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1021/2021] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 04:39:48.607778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc4f6c-4d77-4d05-8370-c8b13f4069b9 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:48.623849 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc4f6c-4d77-4d05-8370-c8b13f4069b9 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:48.624543 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced95c0-5e14-409e-a224-cdc436128c21 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:48.632490 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7109612-0c92-4db7-bec0-80a81aecfee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:48.634612 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7109612-0c92-4db7-bec0-80a81aecfee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:48.637586 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7109612-0c92-4db7-bec0-80a81aecfee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:48.638794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7109612-0c92-4db7-bec0-80a81aecfee1 admin admin] PATH_INFO: `/v3/projects/b3efbf2bb946439d89f790737784f630/users/353cfbda09e34082bbed614067aea039/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:48.641576 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540332e1-fd80-4531-a1e2-9d9a6b9df6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:48.641861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7109612-0c92-4db7-bec0-80a81aecfee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3efbf2bb946439d89f790737784f630, user_id=353cfbda09e34082bbed614067aea039, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:48.642249 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540332e1-fd80-4531-a1e2-9d9a6b9df6db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:48.642530 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540332e1-fd80-4531-a1e2-9d9a6b9df6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:48.642803 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540332e1-fd80-4531-a1e2-9d9a6b9df6db admin admin] PATH_INFO: `/v3/users/cda28cd3d2424868bf57002e388d28af` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:48.644555 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540332e1-fd80-4531-a1e2-9d9a6b9df6db admin admin] RBAC: Authorizing `identity:delete_user(user_id=cda28cd3d2424868bf57002e388d28af)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:48.644719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7109612-0c92-4db7-bec0-80a81aecfee1 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:48.648353 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540332e1-fd80-4531-a1e2-9d9a6b9df6db admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:48.675747 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1022/2022] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:48 2022] PUT /identity/v3/projects/b3efbf2bb946439d89f790737784f630/users/353cfbda09e34082bbed614067aea039/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:48.686727 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d28c2-6cb9-44cc-bcbd-c979207619fc None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:48.687331 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d28c2-6cb9-44cc-bcbd-c979207619fc None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:48.688042 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d28c2-6cb9-44cc-bcbd-c979207619fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:48.733630 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-8b3d28c2-6cb9-44cc-bcbd-c979207619fc None None] MFA Rules not processed for user `353cfbda09e34082bbed614067aea039`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:48.734822 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8b3d28c2-6cb9-44cc-bcbd-c979207619fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:48.743955 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-540332e1-fd80-4531-a1e2-9d9a6b9df6db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cda28cd3d2424868bf57002e388d28af'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:48.744892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-540332e1-fd80-4531-a1e2-9d9a6b9df6db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cda28cd3d2424868bf57002e388d28af'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:48.763613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-540332e1-fd80-4531-a1e2-9d9a6b9df6db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cda28cd3d2424868bf57002e388d28af'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:48.781099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-540332e1-fd80-4531-a1e2-9d9a6b9df6db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cda28cd3d2424868bf57002e388d28af'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:48.781974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-540332e1-fd80-4531-a1e2-9d9a6b9df6db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cda28cd3d2424868bf57002e388d28af'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:48.820283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1001/2023] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:48 2022] DELETE /identity/v3/users/cda28cd3d2424868bf57002e388d28af => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:39:48.820283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540332e1-fd80-4531-a1e2-9d9a6b9df6db admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:48.820283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-540332e1-fd80-4531-a1e2-9d9a6b9df6db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:48.863775 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1023/2024] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:48 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:48.865608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be73e44-828c-4a80-bb9a-76b28620f956 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:48.866022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be73e44-828c-4a80-bb9a-76b28620f956 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:48.866305 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be73e44-828c-4a80-bb9a-76b28620f956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:48.889522 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:48.892066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7be73e44-828c-4a80-bb9a-76b28620f956 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:48.893673 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7be73e44-828c-4a80-bb9a-76b28620f956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:48.920216 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540332e1-fd80-4531-a1e2-9d9a6b9df6db admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:48.926356 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143faf6-07b1-471e-a917-582d053967eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:48.927170 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143faf6-07b1-471e-a917-582d053967eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:48.927454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143faf6-07b1-471e-a917-582d053967eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:48.927728 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143faf6-07b1-471e-a917-582d053967eb admin admin] PATH_INFO: `/v3/projects/ac4ba2ce6e014c00bf7a2be72d85c977` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:48.929231 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143faf6-07b1-471e-a917-582d053967eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac4ba2ce6e014c00bf7a2be72d85c977)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:48.931284 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143faf6-07b1-471e-a917-582d053967eb admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:48.959513 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:48.996021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-7143faf6-07b1-471e-a917-582d053967eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac4ba2ce6e014c00bf7a2be72d85c977', 'request_id': 'req-7143faf6-07b1-471e-a917-582d053967eb'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:48.996961 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1024/2025] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:48.998419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be73e44-828c-4a80-bb9a-76b28620f956 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:49.003182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7be73e44-828c-4a80-bb9a-76b28620f956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:49.008447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-7143faf6-07b1-471e-a917-582d053967eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac4ba2ce6e014c00bf7a2be72d85c977', 'request_id': 'req-7143faf6-07b1-471e-a917-582d053967eb'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:49.017240 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-7143faf6-07b1-471e-a917-582d053967eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac4ba2ce6e014c00bf7a2be72d85c977', 'request_id': 'req-7143faf6-07b1-471e-a917-582d053967eb'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:49.030426 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-7143faf6-07b1-471e-a917-582d053967eb admin admin] The token cache is being invalidate because project ac4ba2ce6e014c00bf7a2be72d85c977 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:49.030572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-7143faf6-07b1-471e-a917-582d053967eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:49.031201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-7143faf6-07b1-471e-a917-582d053967eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:49.037210 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1002/2026] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:48 2022] DELETE /identity/v3/projects/ac4ba2ce6e014c00bf7a2be72d85c977 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:49.037210 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143faf6-07b1-471e-a917-582d053967eb admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:49.037210 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-7143faf6-07b1-471e-a917-582d053967eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:49.129053 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143faf6-07b1-471e-a917-582d053967eb admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:49.133878 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d85609-eb21-4699-b53f-f38592dd12e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:49.134468 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be73e44-828c-4a80-bb9a-76b28620f956 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:49.134579 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d85609-eb21-4699-b53f-f38592dd12e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:49.134902 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d85609-eb21-4699-b53f-f38592dd12e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:49.135185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d85609-eb21-4699-b53f-f38592dd12e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:49.138153 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-54d85609-eb21-4699-b53f-f38592dd12e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:49.271125 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e876e8a-fb2a-4b86-9563-e268b6917fe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:49.272028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e876e8a-fb2a-4b86-9563-e268b6917fe3 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:49.272535 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e876e8a-fb2a-4b86-9563-e268b6917fe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:49.272840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e876e8a-fb2a-4b86-9563-e268b6917fe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:49.279326 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4e876e8a-fb2a-4b86-9563-e268b6917fe3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:49.283577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d85609-eb21-4699-b53f-f38592dd12e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:49.285369 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d85609-eb21-4699-b53f-f38592dd12e8 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:49.297746 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1003/2027] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:49.299093 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cbdd7-0cd6-462b-bfb1-60cd7a533477 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:49.299425 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cbdd7-0cd6-462b-bfb1-60cd7a533477 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:49.299701 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cbdd7-0cd6-462b-bfb1-60cd7a533477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:49.320433 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-f80cbdd7-0cd6-462b-bfb1-60cd7a533477 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:49.321672 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-f80cbdd7-0cd6-462b-bfb1-60cd7a533477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:49.339241 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1004/2028] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:49.341418 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77f30f-0669-4915-863f-0c63d97bfcb7 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:49.341728 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77f30f-0669-4915-863f-0c63d97bfcb7 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:49.342001 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77f30f-0669-4915-863f-0c63d97bfcb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:49.365939 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-ee77f30f-0669-4915-863f-0c63d97bfcb7 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:49.367284 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-ee77f30f-0669-4915-863f-0c63d97bfcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:49.375489 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e876e8a-fb2a-4b86-9563-e268b6917fe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:49.377136 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e876e8a-fb2a-4b86-9563-e268b6917fe3 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:49.381914 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1005/2029] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:49.382841 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee77f30f-0669-4915-863f-0c63d97bfcb7 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:49.388723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:49.390395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1025/2030] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:48 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:49.399688 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee77f30f-0669-4915-863f-0c63d97bfcb7 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:49.410838 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce33e8-c70f-48e3-9eaa-062501864ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:49.411708 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce33e8-c70f-48e3-9eaa-062501864ebb admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:49.412063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce33e8-c70f-48e3-9eaa-062501864ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:49.412513 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce33e8-c70f-48e3-9eaa-062501864ebb admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:49.413348 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce33e8-c70f-48e3-9eaa-062501864ebb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:39:49.414163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce33e8-c70f-48e3-9eaa-062501864ebb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:49.416048 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce33e8-c70f-48e3-9eaa-062501864ebb admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:49.430744 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1006/2031] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:39:49.436258 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e876e8a-fb2a-4b86-9563-e268b6917fe3 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:49.450140 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e876e8a-fb2a-4b86-9563-e268b6917fe3 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:49.455071 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710fd1e-a149-4382-acd7-20cea2ae620b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:49.455698 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710fd1e-a149-4382-acd7-20cea2ae620b admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:49.455983 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710fd1e-a149-4382-acd7-20cea2ae620b admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:49.456257 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710fd1e-a149-4382-acd7-20cea2ae620b admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:49.456935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710fd1e-a149-4382-acd7-20cea2ae620b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:49.458887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710fd1e-a149-4382-acd7-20cea2ae620b admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:49.553312 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce33e8-c70f-48e3-9eaa-062501864ebb admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:49.562871 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1026/2032] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:49 2022] POST /identity/v3/projects => generated 381 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:49.566797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce33e8-c70f-48e3-9eaa-062501864ebb admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:49.669949 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07e86a-18e5-4c6d-add2-d75d009818d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:49.670804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07e86a-18e5-4c6d-add2-d75d009818d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:49.671084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07e86a-18e5-4c6d-add2-d75d009818d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:49.671532 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07e86a-18e5-4c6d-add2-d75d009818d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:49.672286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07e86a-18e5-4c6d-add2-d75d009818d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:49.675053 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07e86a-18e5-4c6d-add2-d75d009818d4 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:49.675730 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710fd1e-a149-4382-acd7-20cea2ae620b admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:49.677764 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1710fd1e-a149-4382-acd7-20cea2ae620b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:49.709053 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1007/2033] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:49 2022] POST /identity/v3/users => generated 291 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:49.713403 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07e86a-18e5-4c6d-add2-d75d009818d4 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:49.728925 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07e86a-18e5-4c6d-add2-d75d009818d4 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:49.736397 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1484f8c-9870-4871-84d8-aacbfc861984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:49.737171 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1484f8c-9870-4871-84d8-aacbfc861984 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:49.737517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1484f8c-9870-4871-84d8-aacbfc861984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:49.737948 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1484f8c-9870-4871-84d8-aacbfc861984 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:49.738774 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1484f8c-9870-4871-84d8-aacbfc861984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:49.740659 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1484f8c-9870-4871-84d8-aacbfc861984 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:49.744202 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:49.757516 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1008/2034] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 04:39:49.757516 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1484f8c-9870-4871-84d8-aacbfc861984 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:49.760168 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-a1484f8c-9870-4871-84d8-aacbfc861984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:49.776590 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1484f8c-9870-4871-84d8-aacbfc861984 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:49.781832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710fd1e-a149-4382-acd7-20cea2ae620b admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:49.783372 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b3961-0757-4294-b9e6-2297f68d91e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:49.784159 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b3961-0757-4294-b9e6-2297f68d91e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:49.784536 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b3961-0757-4294-b9e6-2297f68d91e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:49.784877 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b3961-0757-4294-b9e6-2297f68d91e1 admin admin] PATH_INFO: `/v3/projects/2db11129a2384e79af22b323ad1f08f2` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:49.786017 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05b8f5-fb22-4f89-9ea5-ad682d49995d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:49.786324 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b3961-0757-4294-b9e6-2297f68d91e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2db11129a2384e79af22b323ad1f08f2)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:49.786639 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05b8f5-fb22-4f89-9ea5-ad682d49995d service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:49.786919 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05b8f5-fb22-4f89-9ea5-ad682d49995d service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:49.787191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05b8f5-fb22-4f89-9ea5-ad682d49995d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:49.788739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b3961-0757-4294-b9e6-2297f68d91e1 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:49.789043 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9c05b8f5-fb22-4f89-9ea5-ad682d49995d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:49.881418 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-347b3961-0757-4294-b9e6-2297f68d91e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2db11129a2384e79af22b323ad1f08f2', 'request_id': 'req-347b3961-0757-4294-b9e6-2297f68d91e1'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:49.892380 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c05b8f5-fb22-4f89-9ea5-ad682d49995d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:49.894058 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c05b8f5-fb22-4f89-9ea5-ad682d49995d service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:49.896369 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-347b3961-0757-4294-b9e6-2297f68d91e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2db11129a2384e79af22b323ad1f08f2', 'request_id': 'req-347b3961-0757-4294-b9e6-2297f68d91e1'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:49.905030 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1027/2035] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:49 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:39:49.906570 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-347b3961-0757-4294-b9e6-2297f68d91e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2db11129a2384e79af22b323ad1f08f2', 'request_id': 'req-347b3961-0757-4294-b9e6-2297f68d91e1'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:49.916528 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05b8f5-fb22-4f89-9ea5-ad682d49995d service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:49.927009 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-347b3961-0757-4294-b9e6-2297f68d91e1 admin admin] The token cache is being invalidate because project 2db11129a2384e79af22b323ad1f08f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:49.927411 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-347b3961-0757-4294-b9e6-2297f68d91e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:49.928370 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-347b3961-0757-4294-b9e6-2297f68d91e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:49.933212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1009/2036] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:49 2022] DELETE /identity/v3/projects/2db11129a2384e79af22b323ad1f08f2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:49.937209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05b8f5-fb22-4f89-9ea5-ad682d49995d service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:49.937967 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b3961-0757-4294-b9e6-2297f68d91e1 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:49.941064 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-347b3961-0757-4294-b9e6-2297f68d91e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:50.039276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28fba9-1aa4-4464-9af0-e9bd7680b2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:50.039947 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28fba9-1aa4-4464-9af0-e9bd7680b2e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:50.040206 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28fba9-1aa4-4464-9af0-e9bd7680b2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:50.040482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28fba9-1aa4-4464-9af0-e9bd7680b2e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:50.041610 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28fba9-1aa4-4464-9af0-e9bd7680b2e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:50.044240 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28fba9-1aa4-4464-9af0-e9bd7680b2e9 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:50.044488 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:50.057209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1028/2037] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:50.062153 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28fba9-1aa4-4464-9af0-e9bd7680b2e9 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:50.063774 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b3961-0757-4294-b9e6-2297f68d91e1 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:50.063991 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fb28fba9-1aa4-4464-9af0-e9bd7680b2e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:50.072896 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211b6a5-559b-4260-9cd0-cefe72e3e3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:50.073880 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211b6a5-559b-4260-9cd0-cefe72e3e3bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:50.075003 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211b6a5-559b-4260-9cd0-cefe72e3e3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:50.075385 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211b6a5-559b-4260-9cd0-cefe72e3e3bc admin admin] PATH_INFO: `/v3/users/09ad26801c6f49f29e5ca2e49dc935fc` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:50.077018 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211b6a5-559b-4260-9cd0-cefe72e3e3bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=09ad26801c6f49f29e5ca2e49dc935fc)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:50.078743 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28fba9-1aa4-4464-9af0-e9bd7680b2e9 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:50.081382 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211b6a5-559b-4260-9cd0-cefe72e3e3bc admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:50.083295 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd84c1-a411-47b4-8af5-9a679c1d77da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:50.083969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd84c1-a411-47b4-8af5-9a679c1d77da admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:50.084242 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd84c1-a411-47b4-8af5-9a679c1d77da admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:50.084587 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd84c1-a411-47b4-8af5-9a679c1d77da admin admin] PATH_INFO: `/v3/projects/5d0a84231ad44db1af3082bd50d04e85/users/c75210a54f4142ca90529efa3d9b9e95/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:50.105606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd84c1-a411-47b4-8af5-9a679c1d77da admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d0a84231ad44db1af3082bd50d04e85, user_id=c75210a54f4142ca90529efa3d9b9e95, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:50.109761 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd84c1-a411-47b4-8af5-9a679c1d77da admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:50.131018 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1029/2038] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:50 2022] PUT /identity/v3/projects/5d0a84231ad44db1af3082bd50d04e85/users/c75210a54f4142ca90529efa3d9b9e95/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:39:50.145081 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-6211b6a5-559b-4260-9cd0-cefe72e3e3bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09ad26801c6f49f29e5ca2e49dc935fc'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:50.146009 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc216a-8a32-4af6-9cc8-648832a251a3 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:50.146162 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-6211b6a5-559b-4260-9cd0-cefe72e3e3bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09ad26801c6f49f29e5ca2e49dc935fc'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:50.146359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc216a-8a32-4af6-9cc8-648832a251a3 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:50.146716 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc216a-8a32-4af6-9cc8-648832a251a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:50.166822 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-6211b6a5-559b-4260-9cd0-cefe72e3e3bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09ad26801c6f49f29e5ca2e49dc935fc'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:50.177563 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-6211b6a5-559b-4260-9cd0-cefe72e3e3bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09ad26801c6f49f29e5ca2e49dc935fc'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:50.178329 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-6211b6a5-559b-4260-9cd0-cefe72e3e3bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09ad26801c6f49f29e5ca2e49dc935fc'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:50.180021 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b2bc216a-8a32-4af6-9cc8-648832a251a3 None None] MFA Rules not processed for user `c75210a54f4142ca90529efa3d9b9e95`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:50.181341 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b2bc216a-8a32-4af6-9cc8-648832a251a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:50.196084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1010/2039] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:49 2022] DELETE /identity/v3/users/09ad26801c6f49f29e5ca2e49dc935fc => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:50.241418 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:50.281215 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1030/2040] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:50.295059 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211b6a5-559b-4260-9cd0-cefe72e3e3bc admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:50.297260 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-6211b6a5-559b-4260-9cd0-cefe72e3e3bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:50.400025 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211b6a5-559b-4260-9cd0-cefe72e3e3bc admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:50.405477 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f01f4-4968-44cb-9e25-6eb53cd1492e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:50.406145 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f01f4-4968-44cb-9e25-6eb53cd1492e admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:50.406480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f01f4-4968-44cb-9e25-6eb53cd1492e admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:50.406759 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f01f4-4968-44cb-9e25-6eb53cd1492e admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:50.407446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f01f4-4968-44cb-9e25-6eb53cd1492e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:50.409460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f01f4-4968-44cb-9e25-6eb53cd1492e admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:50.444185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1011/2041] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:50 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:50.448241 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc216a-8a32-4af6-9cc8-648832a251a3 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:50.529252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc216a-8a32-4af6-9cc8-648832a251a3 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:50.571945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f01f4-4968-44cb-9e25-6eb53cd1492e admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:50.574464 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-e02f01f4-4968-44cb-9e25-6eb53cd1492e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:50.608039 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386dfff-d45b-4781-bd99-da608bf78cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:50.608949 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386dfff-d45b-4781-bd99-da608bf78cc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:50.609176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386dfff-d45b-4781-bd99-da608bf78cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:50.609500 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386dfff-d45b-4781-bd99-da608bf78cc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:50.610239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b386dfff-d45b-4781-bd99-da608bf78cc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:50.612321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b386dfff-d45b-4781-bd99-da608bf78cc4 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:50.636388 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1031/2042] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:50 2022] POST /identity/v3/users => generated 298 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 04:39:50.640001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386dfff-d45b-4781-bd99-da608bf78cc4 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:50.649825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:50.653408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386dfff-d45b-4781-bd99-da608bf78cc4 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:50.660344 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f01f4-4968-44cb-9e25-6eb53cd1492e admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:50.660447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7195d-a6fa-4d0c-8df2-3f4bbed5e100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:50.662426 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7195d-a6fa-4d0c-8df2-3f4bbed5e100 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:50.662426 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7195d-a6fa-4d0c-8df2-3f4bbed5e100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:50.662763 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7195d-a6fa-4d0c-8df2-3f4bbed5e100 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:50.664227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7195d-a6fa-4d0c-8df2-3f4bbed5e100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:50.669194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7195d-a6fa-4d0c-8df2-3f4bbed5e100 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:50.675307 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769dca3-2b24-4470-97a9-da5393eeb502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:50.676062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769dca3-2b24-4470-97a9-da5393eeb502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:50.677137 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769dca3-2b24-4470-97a9-da5393eeb502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:50.677866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769dca3-2b24-4470-97a9-da5393eeb502 admin admin] PATH_INFO: `/v3/projects/b6e32d2a6b3344ddb02a6038ced90c66` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:50.680056 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769dca3-2b24-4470-97a9-da5393eeb502 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6e32d2a6b3344ddb02a6038ced90c66)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:50.684142 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769dca3-2b24-4470-97a9-da5393eeb502 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:50.686071 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1032/2043] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 04:39:50.690895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7195d-a6fa-4d0c-8df2-3f4bbed5e100 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:50.702729 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7195d-a6fa-4d0c-8df2-3f4bbed5e100 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:50.725991 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3fc66-c438-4cb6-9a45-81145744934f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:50.727180 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3fc66-c438-4cb6-9a45-81145744934f admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:50.728844 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3fc66-c438-4cb6-9a45-81145744934f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:50.729132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3fc66-c438-4cb6-9a45-81145744934f admin admin] PATH_INFO: `/v3/projects/b3367b2540374bc0867934cc95856d0b/users/aef636f2543642338f86f01e54e42430/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:50.763856 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3fc66-c438-4cb6-9a45-81145744934f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3367b2540374bc0867934cc95856d0b, user_id=aef636f2543642338f86f01e54e42430, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:50.766126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5769dca3-2b24-4470-97a9-da5393eeb502 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6e32d2a6b3344ddb02a6038ced90c66', 'request_id': 'req-5769dca3-2b24-4470-97a9-da5393eeb502'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:50.767174 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3fc66-c438-4cb6-9a45-81145744934f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:50.791652 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1033/2044] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:50 2022] PUT /identity/v3/projects/b3367b2540374bc0867934cc95856d0b/users/aef636f2543642338f86f01e54e42430/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:50.792036 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5769dca3-2b24-4470-97a9-da5393eeb502 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6e32d2a6b3344ddb02a6038ced90c66', 'request_id': 'req-5769dca3-2b24-4470-97a9-da5393eeb502'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:50.792777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3fc66-c438-4cb6-9a45-81145744934f admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:50.805720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5769dca3-2b24-4470-97a9-da5393eeb502 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6e32d2a6b3344ddb02a6038ced90c66', 'request_id': 'req-5769dca3-2b24-4470-97a9-da5393eeb502'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:50.822239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5769dca3-2b24-4470-97a9-da5393eeb502 admin admin] The token cache is being invalidate because project b6e32d2a6b3344ddb02a6038ced90c66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:50.822707 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5769dca3-2b24-4470-97a9-da5393eeb502 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:50.824281 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5769dca3-2b24-4470-97a9-da5393eeb502 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:50.829669 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1012/2045] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:50 2022] DELETE /identity/v3/projects/b6e32d2a6b3344ddb02a6038ced90c66 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:50.832961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f5df5-b495-4bba-aa81-12f959bbc0e3 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:50.833244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f5df5-b495-4bba-aa81-12f959bbc0e3 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:50.833938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f5df5-b495-4bba-aa81-12f959bbc0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:50.862278 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-ed8f5df5-b495-4bba-aa81-12f959bbc0e3 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:50.863525 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-ed8f5df5-b495-4bba-aa81-12f959bbc0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:50.872506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:50.913939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3fc66-c438-4cb6-9a45-81145744934f admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:50.926550 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894db704-0f22-43e7-9eaa-fcdf4f149826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:50.927205 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894db704-0f22-43e7-9eaa-fcdf4f149826 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:50.927623 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894db704-0f22-43e7-9eaa-fcdf4f149826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:50.928319 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894db704-0f22-43e7-9eaa-fcdf4f149826 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:50.929237 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894db704-0f22-43e7-9eaa-fcdf4f149826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:50.933426 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894db704-0f22-43e7-9eaa-fcdf4f149826 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:50.970172 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1034/2046] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:50.988228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894db704-0f22-43e7-9eaa-fcdf4f149826 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:50.988659 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1013/2047] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:50.997451 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb901986-247f-46cb-bc08-2efb14c5024a None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:50.997825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb901986-247f-46cb-bc08-2efb14c5024a None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:50.998121 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb901986-247f-46cb-bc08-2efb14c5024a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:50.999487 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-894db704-0f22-43e7-9eaa-fcdf4f149826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:51.020964 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-fb901986-247f-46cb-bc08-2efb14c5024a None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:51.022407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-fb901986-247f-46cb-bc08-2efb14c5024a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:51.036432 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1014/2048] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:51.037535 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb901986-247f-46cb-bc08-2efb14c5024a None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:51.048658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb901986-247f-46cb-bc08-2efb14c5024a None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:51.054325 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd8289-62e4-486e-a293-e542f131e802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:51.055066 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd8289-62e4-486e-a293-e542f131e802 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:51.055349 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd8289-62e4-486e-a293-e542f131e802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:51.055651 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd8289-62e4-486e-a293-e542f131e802 admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:51.055772 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894db704-0f22-43e7-9eaa-fcdf4f149826 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:51.056367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd8289-62e4-486e-a293-e542f131e802 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:39:51.056668 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd8289-62e4-486e-a293-e542f131e802 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:51.058328 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd8289-62e4-486e-a293-e542f131e802 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:51.065373 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcbc458-2fbf-4ca6-a89d-13fafed522b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:51.066364 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbc458-2fbf-4ca6-a89d-13fafed522b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:51.066720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbc458-2fbf-4ca6-a89d-13fafed522b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:51.067387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbc458-2fbf-4ca6-a89d-13fafed522b1 admin admin] PATH_INFO: `/v3/projects/b3367b2540374bc0867934cc95856d0b/users/aef636f2543642338f86f01e54e42430/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:51.076524 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1015/2049] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:39:51.077479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd8289-62e4-486e-a293-e542f131e802 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:51.086374 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcbc458-2fbf-4ca6-a89d-13fafed522b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3367b2540374bc0867934cc95856d0b, user_id=aef636f2543642338f86f01e54e42430, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:51.087932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd8289-62e4-486e-a293-e542f131e802 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:51.093039 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd51f17-3f1a-4fe1-9049-046b7fc7affc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:51.093807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd51f17-3f1a-4fe1-9049-046b7fc7affc admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:51.094112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd51f17-3f1a-4fe1-9049-046b7fc7affc admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:51.094501 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd51f17-3f1a-4fe1-9049-046b7fc7affc admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:51.095281 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd51f17-3f1a-4fe1-9049-046b7fc7affc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:51.097453 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd51f17-3f1a-4fe1-9049-046b7fc7affc admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:51.112089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcbc458-2fbf-4ca6-a89d-13fafed522b1 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:51.137764 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1035/2050] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:50 2022] PUT /identity/v3/projects/b3367b2540374bc0867934cc95856d0b/users/aef636f2543642338f86f01e54e42430/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:51.150339 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754361df-e1dd-42c5-9968-3578e70c035c None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:51.150726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754361df-e1dd-42c5-9968-3578e70c035c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:51.150953 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754361df-e1dd-42c5-9968-3578e70c035c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:51.159588 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1016/2051] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:51 2022] POST /identity/v3/projects => generated 381 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:51.163745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd51f17-3f1a-4fe1-9049-046b7fc7affc admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:51.204651 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-754361df-e1dd-42c5-9968-3578e70c035c None None] MFA Rules not processed for user `aef636f2543642338f86f01e54e42430`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:51.206107 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-754361df-e1dd-42c5-9968-3578e70c035c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:51.239429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:51.264089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd51f17-3f1a-4fe1-9049-046b7fc7affc admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:51.269400 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d7720-e0cf-4d42-be83-4d51db589f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:51.270097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d7720-e0cf-4d42-be83-4d51db589f76 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:51.270380 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d7720-e0cf-4d42-be83-4d51db589f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:51.270657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d7720-e0cf-4d42-be83-4d51db589f76 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:51.271341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34d7720-e0cf-4d42-be83-4d51db589f76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:51.273273 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34d7720-e0cf-4d42-be83-4d51db589f76 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:51.294815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1036/2052] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:51 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:51.306077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1017/2053] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:51 2022] POST /identity/v3/users => generated 291 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:51.306607 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754361df-e1dd-42c5-9968-3578e70c035c None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:51.311919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d7720-e0cf-4d42-be83-4d51db589f76 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:51.312169 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-754361df-e1dd-42c5-9968-3578e70c035c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:51.323531 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d7720-e0cf-4d42-be83-4d51db589f76 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:51.351245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b31be-b38b-4ade-af3d-da98d1180ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:51.351894 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b31be-b38b-4ade-af3d-da98d1180ace admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:51.352182 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b31be-b38b-4ade-af3d-da98d1180ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:51.352458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b31be-b38b-4ade-af3d-da98d1180ace admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:51.353177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b31be-b38b-4ade-af3d-da98d1180ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:51.354970 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b31be-b38b-4ade-af3d-da98d1180ace admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:51.370441 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1018/2054] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 04:39:51.376252 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b31be-b38b-4ade-af3d-da98d1180ace admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:51.389316 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b31be-b38b-4ade-af3d-da98d1180ace admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:51.396323 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc73b79-885b-415f-8b34-c14a0e754e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:51.396977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc73b79-885b-415f-8b34-c14a0e754e2e admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:51.397433 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc73b79-885b-415f-8b34-c14a0e754e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:51.397767 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc73b79-885b-415f-8b34-c14a0e754e2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:51.400053 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc73b79-885b-415f-8b34-c14a0e754e2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:51.402737 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc73b79-885b-415f-8b34-c14a0e754e2e admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:51.425844 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1019/2055] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:51.427522 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc73b79-885b-415f-8b34-c14a0e754e2e admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:51.436793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754361df-e1dd-42c5-9968-3578e70c035c None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:51.441280 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00368199-3f3c-4d7d-b119-5eeb8e4a0e68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:51.441958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00368199-3f3c-4d7d-b119-5eeb8e4a0e68 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:51.442241 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00368199-3f3c-4d7d-b119-5eeb8e4a0e68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:51.442513 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00368199-3f3c-4d7d-b119-5eeb8e4a0e68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:51.445192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc73b79-885b-415f-8b34-c14a0e754e2e admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:51.454508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4def20d-d24a-4e03-92a9-298298f82c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:51.455565 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00368199-3f3c-4d7d-b119-5eeb8e4a0e68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:51.455686 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4def20d-d24a-4e03-92a9-298298f82c83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:51.455971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4def20d-d24a-4e03-92a9-298298f82c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:51.456263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4def20d-d24a-4e03-92a9-298298f82c83 admin admin] PATH_INFO: `/v3/projects/0c542fb4e5634418bcf3070f60c98fdb/users/fd7b0ed26cc544fd9774877e848cc046/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:51.457419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00368199-3f3c-4d7d-b119-5eeb8e4a0e68 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:51.476885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1037/2056] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:51 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:39:51.489942 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4def20d-d24a-4e03-92a9-298298f82c83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c542fb4e5634418bcf3070f60c98fdb, user_id=fd7b0ed26cc544fd9774877e848cc046, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:51.497298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4def20d-d24a-4e03-92a9-298298f82c83 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:51.537191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1020/2057] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:51 2022] PUT /identity/v3/projects/0c542fb4e5634418bcf3070f60c98fdb/users/fd7b0ed26cc544fd9774877e848cc046/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:51.557971 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23052825-0dec-454e-962f-3ec9bc1bb8b5 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:51.557971 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23052825-0dec-454e-962f-3ec9bc1bb8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:51.557971 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23052825-0dec-454e-962f-3ec9bc1bb8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:51.640777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-23052825-0dec-454e-962f-3ec9bc1bb8b5 None None] MFA Rules not processed for user `fd7b0ed26cc544fd9774877e848cc046`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:51.642345 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-23052825-0dec-454e-962f-3ec9bc1bb8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:51.763933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:51.817582 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4def20d-d24a-4e03-92a9-298298f82c83 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:51.819510 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c4def20d-d24a-4e03-92a9-298298f82c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:51.832744 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1038/2058] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:51.837177 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23052825-0dec-454e-962f-3ec9bc1bb8b5 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:51.963409 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4def20d-d24a-4e03-92a9-298298f82c83 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:51.968654 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23052825-0dec-454e-962f-3ec9bc1bb8b5 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:51.971556 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e8eda-c961-4eb2-9ebe-eb00162a911f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:51.972804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e8eda-c961-4eb2-9ebe-eb00162a911f service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:51.973312 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e8eda-c961-4eb2-9ebe-eb00162a911f service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:51.973845 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e8eda-c961-4eb2-9ebe-eb00162a911f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:51.977590 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-581e8eda-c961-4eb2-9ebe-eb00162a911f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:51.978394 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daceac0d-05d0-4bfe-be76-383e3fea2933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:51.979339 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daceac0d-05d0-4bfe-be76-383e3fea2933 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:51.979778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daceac0d-05d0-4bfe-be76-383e3fea2933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:51.980931 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daceac0d-05d0-4bfe-be76-383e3fea2933 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:51.981938 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daceac0d-05d0-4bfe-be76-383e3fea2933 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:51.991804 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daceac0d-05d0-4bfe-be76-383e3fea2933 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:52.043179 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1039/2059] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:51 2022] POST /identity/v3/projects => generated 379 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:52.050515 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daceac0d-05d0-4bfe-be76-383e3fea2933 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:52.107080 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e8eda-c961-4eb2-9ebe-eb00162a911f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:52.113866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e8eda-c961-4eb2-9ebe-eb00162a911f service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:52.147571 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1021/2060] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:39:52.187517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daceac0d-05d0-4bfe-be76-383e3fea2933 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:52.196320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12582898-9d2d-441a-b466-1f934573b461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:52.197655 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12582898-9d2d-441a-b466-1f934573b461 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:52.200259 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12582898-9d2d-441a-b466-1f934573b461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:52.201065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12582898-9d2d-441a-b466-1f934573b461 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:52.202222 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12582898-9d2d-441a-b466-1f934573b461 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:52.205411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12582898-9d2d-441a-b466-1f934573b461 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:52.237486 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1040/2061] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:52 2022] POST /identity/v3/users => generated 296 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:52.241627 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e8eda-c961-4eb2-9ebe-eb00162a911f service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:52.260609 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e8eda-c961-4eb2-9ebe-eb00162a911f service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:52.266681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19327b7a-395c-48d3-bc8e-f6cc248ca6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:52.267402 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19327b7a-395c-48d3-bc8e-f6cc248ca6ce admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:52.267787 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19327b7a-395c-48d3-bc8e-f6cc248ca6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:52.268116 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19327b7a-395c-48d3-bc8e-f6cc248ca6ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:52.269094 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19327b7a-395c-48d3-bc8e-f6cc248ca6ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:52.270976 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19327b7a-395c-48d3-bc8e-f6cc248ca6ce admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:52.288465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1022/2062] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 04:39:52.292672 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12582898-9d2d-441a-b466-1f934573b461 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:52.316590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12582898-9d2d-441a-b466-1f934573b461 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:52.329318 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7e5a6-b0e4-42d1-a7cf-148fd2d1b9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:52.330237 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7e5a6-b0e4-42d1-a7cf-148fd2d1b9fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:52.330491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7e5a6-b0e4-42d1-a7cf-148fd2d1b9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:52.330764 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7e5a6-b0e4-42d1-a7cf-148fd2d1b9fb admin admin] PATH_INFO: `/v3/projects/e0b51e63301048fab44ee19fbd823261/users/98c5f8909de241ecae28fa43a7ec27c5/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:52.351409 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7e5a6-b0e4-42d1-a7cf-148fd2d1b9fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0b51e63301048fab44ee19fbd823261, user_id=98c5f8909de241ecae28fa43a7ec27c5, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:52.354225 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7e5a6-b0e4-42d1-a7cf-148fd2d1b9fb admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:52.372918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1041/2063] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:52 2022] PUT /identity/v3/projects/e0b51e63301048fab44ee19fbd823261/users/98c5f8909de241ecae28fa43a7ec27c5/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:52.377082 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19327b7a-395c-48d3-bc8e-f6cc248ca6ce admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:52.388606 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19327b7a-395c-48d3-bc8e-f6cc248ca6ce admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:52.425368 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:52.456970 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf897c4-7f6b-4b8f-88eb-188d43344aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:52.457514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf897c4-7f6b-4b8f-88eb-188d43344aae admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:52.457848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf897c4-7f6b-4b8f-88eb-188d43344aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:52.458127 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf897c4-7f6b-4b8f-88eb-188d43344aae admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:52.458841 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf897c4-7f6b-4b8f-88eb-188d43344aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:52.460471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf897c4-7f6b-4b8f-88eb-188d43344aae admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:52.472908 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1023/2064] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:52.477482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7e5a6-b0e4-42d1-a7cf-148fd2d1b9fb admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:52.488419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7e5a6-b0e4-42d1-a7cf-148fd2d1b9fb admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:52.493279 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4444b-3f76-4cb1-a5c4-bcfd0f0af16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:52.494082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4444b-3f76-4cb1-a5c4-bcfd0f0af16f admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:52.494488 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4444b-3f76-4cb1-a5c4-bcfd0f0af16f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:52.494822 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4444b-3f76-4cb1-a5c4-bcfd0f0af16f admin admin] PATH_INFO: `/v3/projects/e0b51e63301048fab44ee19fbd823261/users/98c5f8909de241ecae28fa43a7ec27c5/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:52.498270 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b4444b-3f76-4cb1-a5c4-bcfd0f0af16f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0b51e63301048fab44ee19fbd823261, user_id=98c5f8909de241ecae28fa43a7ec27c5, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:52.499320 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf897c4-7f6b-4b8f-88eb-188d43344aae admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:52.500186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b4444b-3f76-4cb1-a5c4-bcfd0f0af16f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:52.516532 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf897c4-7f6b-4b8f-88eb-188d43344aae admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:52.521453 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1042/2065] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:52 2022] PUT /identity/v3/projects/e0b51e63301048fab44ee19fbd823261/users/98c5f8909de241ecae28fa43a7ec27c5/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:39:52.533222 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0169f5-38a1-41a0-ade5-87e37e9fb4c8 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:52.533750 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0169f5-38a1-41a0-ade5-87e37e9fb4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:52.534205 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0169f5-38a1-41a0-ade5-87e37e9fb4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:52.563906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-dd0169f5-38a1-41a0-ade5-87e37e9fb4c8 None None] MFA Rules not processed for user `98c5f8909de241ecae28fa43a7ec27c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:52.565143 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-dd0169f5-38a1-41a0-ade5-87e37e9fb4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:52.576706 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb05e3-1f6c-4d71-ab89-f280b4a55a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:52.577374 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb05e3-1f6c-4d71-ab89-f280b4a55a01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:52.577588 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:52.577768 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb05e3-1f6c-4d71-ab89-f280b4a55a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:52.577934 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb05e3-1f6c-4d71-ab89-f280b4a55a01 admin admin] PATH_INFO: `/v3/users/c75210a54f4142ca90529efa3d9b9e95` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:52.579354 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb05e3-1f6c-4d71-ab89-f280b4a55a01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c75210a54f4142ca90529efa3d9b9e95)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:52.581844 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb05e3-1f6c-4d71-ab89-f280b4a55a01 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:52.640750 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-3feb05e3-1f6c-4d71-ab89-f280b4a55a01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c75210a54f4142ca90529efa3d9b9e95'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:52.641486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-3feb05e3-1f6c-4d71-ab89-f280b4a55a01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c75210a54f4142ca90529efa3d9b9e95'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:52.651144 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1043/2066] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:52 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:39:52.656578 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-3feb05e3-1f6c-4d71-ab89-f280b4a55a01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c75210a54f4142ca90529efa3d9b9e95'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:52.662047 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0169f5-38a1-41a0-ade5-87e37e9fb4c8 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:52.665198 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-3feb05e3-1f6c-4d71-ab89-f280b4a55a01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c75210a54f4142ca90529efa3d9b9e95'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:52.665865 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-3feb05e3-1f6c-4d71-ab89-f280b4a55a01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c75210a54f4142ca90529efa3d9b9e95'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:52.681384 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1024/2067] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:52 2022] DELETE /identity/v3/users/c75210a54f4142ca90529efa3d9b9e95 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:39:52.687211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fee0f5-078d-45db-a2c7-824d055f958b None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:52.687554 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fee0f5-078d-45db-a2c7-824d055f958b None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:52.687838 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fee0f5-078d-45db-a2c7-824d055f958b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:52.708223 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-59fee0f5-078d-45db-a2c7-824d055f958b None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:52.709634 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-59fee0f5-078d-45db-a2c7-824d055f958b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:52.724104 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0169f5-38a1-41a0-ade5-87e37e9fb4c8 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:52.757334 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:52.775463 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1025/2068] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:39:52.785825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fee0f5-078d-45db-a2c7-824d055f958b None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:52.787446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-59fee0f5-078d-45db-a2c7-824d055f958b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:52.792123 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74d061-5c27-411e-a12e-f68b3d95ae70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:52.793411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74d061-5c27-411e-a12e-f68b3d95ae70 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:52.794008 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74d061-5c27-411e-a12e-f68b3d95ae70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:52.794346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74d061-5c27-411e-a12e-f68b3d95ae70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:52.796218 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-5e74d061-5c27-411e-a12e-f68b3d95ae70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:52.847960 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fee0f5-078d-45db-a2c7-824d055f958b None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:52.851887 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d184c-f621-43d4-b9f6-269c7773ef43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:52.852586 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d184c-f621-43d4-b9f6-269c7773ef43 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:52.852942 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d184c-f621-43d4-b9f6-269c7773ef43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:52.853289 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d184c-f621-43d4-b9f6-269c7773ef43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:52.864242 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d184c-f621-43d4-b9f6-269c7773ef43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:52.865833 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d184c-f621-43d4-b9f6-269c7773ef43 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:52.865968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74d061-5c27-411e-a12e-f68b3d95ae70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:52.867445 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74d061-5c27-411e-a12e-f68b3d95ae70 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:52.876487 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1026/2069] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:39:52.879175 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1044/2070] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:52 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:39:53.206657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d184c-f621-43d4-b9f6-269c7773ef43 service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:53.208287 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-990d184c-f621-43d4-b9f6-269c7773ef43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:53.226907 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d184c-f621-43d4-b9f6-269c7773ef43 service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:53.232290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de4173f-760f-4fb8-8d74-6a4081da5882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:53.233042 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4173f-760f-4fb8-8d74-6a4081da5882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:53.233389 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4173f-760f-4fb8-8d74-6a4081da5882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:53.233722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4173f-760f-4fb8-8d74-6a4081da5882 admin admin] PATH_INFO: `/v3/projects/5d0a84231ad44db1af3082bd50d04e85` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:53.234908 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de4173f-760f-4fb8-8d74-6a4081da5882 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d0a84231ad44db1af3082bd50d04e85)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:53.237257 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de4173f-760f-4fb8-8d74-6a4081da5882 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:53.307447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-3de4173f-760f-4fb8-8d74-6a4081da5882 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d0a84231ad44db1af3082bd50d04e85', 'request_id': 'req-3de4173f-760f-4fb8-8d74-6a4081da5882'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:53.319987 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-3de4173f-760f-4fb8-8d74-6a4081da5882 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d0a84231ad44db1af3082bd50d04e85', 'request_id': 'req-3de4173f-760f-4fb8-8d74-6a4081da5882'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:53.329460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-3de4173f-760f-4fb8-8d74-6a4081da5882 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d0a84231ad44db1af3082bd50d04e85', 'request_id': 'req-3de4173f-760f-4fb8-8d74-6a4081da5882'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:53.351246 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-3de4173f-760f-4fb8-8d74-6a4081da5882 admin admin] The token cache is being invalidate because project 5d0a84231ad44db1af3082bd50d04e85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:53.351542 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-3de4173f-760f-4fb8-8d74-6a4081da5882 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:53.352196 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-3de4173f-760f-4fb8-8d74-6a4081da5882 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:53.356863 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1027/2071] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:53 2022] DELETE /identity/v3/projects/5d0a84231ad44db1af3082bd50d04e85 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:53.358068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74d061-5c27-411e-a12e-f68b3d95ae70 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:53.366378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de4173f-760f-4fb8-8d74-6a4081da5882 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:53.367996 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-3de4173f-760f-4fb8-8d74-6a4081da5882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:53.370955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74d061-5c27-411e-a12e-f68b3d95ae70 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:53.384806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:53.450402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78578e15-5153-4628-8d2d-98333dd37dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:53.451525 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78578e15-5153-4628-8d2d-98333dd37dc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:53.451995 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78578e15-5153-4628-8d2d-98333dd37dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:53.452301 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78578e15-5153-4628-8d2d-98333dd37dc0 admin admin] PATH_INFO: `/v3/users/aef636f2543642338f86f01e54e42430` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:53.453907 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78578e15-5153-4628-8d2d-98333dd37dc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aef636f2543642338f86f01e54e42430)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:53.455897 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78578e15-5153-4628-8d2d-98333dd37dc0 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:53.462925 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de4173f-760f-4fb8-8d74-6a4081da5882 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:53.469212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da12d1-e060-4608-8faf-74d05fdaa7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:53.469932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da12d1-e060-4608-8faf-74d05fdaa7ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:53.470211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da12d1-e060-4608-8faf-74d05fdaa7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:53.470486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da12d1-e060-4608-8faf-74d05fdaa7ca admin admin] PATH_INFO: `/v3/users/fd7b0ed26cc544fd9774877e848cc046` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:53.471903 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da12d1-e060-4608-8faf-74d05fdaa7ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd7b0ed26cc544fd9774877e848cc046)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:53.474255 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da12d1-e060-4608-8faf-74d05fdaa7ca admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:53.493366 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:53.523101 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-78578e15-5153-4628-8d2d-98333dd37dc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aef636f2543642338f86f01e54e42430'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:53.532913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-78578e15-5153-4628-8d2d-98333dd37dc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aef636f2543642338f86f01e54e42430'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:53.533688 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-78578e15-5153-4628-8d2d-98333dd37dc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aef636f2543642338f86f01e54e42430'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:53.542631 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-91da12d1-e060-4608-8faf-74d05fdaa7ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd7b0ed26cc544fd9774877e848cc046'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:53.543337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-91da12d1-e060-4608-8faf-74d05fdaa7ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd7b0ed26cc544fd9774877e848cc046'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:53.547438 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-78578e15-5153-4628-8d2d-98333dd37dc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aef636f2543642338f86f01e54e42430'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:53.548128 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-78578e15-5153-4628-8d2d-98333dd37dc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aef636f2543642338f86f01e54e42430'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:53.559149 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-91da12d1-e060-4608-8faf-74d05fdaa7ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd7b0ed26cc544fd9774877e848cc046'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:53.564167 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1045/2072] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:53 2022] DELETE /identity/v3/users/aef636f2543642338f86f01e54e42430 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:39:53.570831 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-91da12d1-e060-4608-8faf-74d05fdaa7ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd7b0ed26cc544fd9774877e848cc046'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:53.571478 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-91da12d1-e060-4608-8faf-74d05fdaa7ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd7b0ed26cc544fd9774877e848cc046'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:53.586638 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1028/2073] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:53 2022] DELETE /identity/v3/users/fd7b0ed26cc544fd9774877e848cc046 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:39:53.592742 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d27964-8ff1-4b93-bc41-126342066e94 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:53.593029 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d27964-8ff1-4b93-bc41-126342066e94 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:53.593458 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d27964-8ff1-4b93-bc41-126342066e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:53.615037 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-26d27964-8ff1-4b93-bc41-126342066e94 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:53.616260 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-26d27964-8ff1-4b93-bc41-126342066e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:53.682578 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1046/2074] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:53.693031 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da12d1-e060-4608-8faf-74d05fdaa7ca admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:53.695945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-91da12d1-e060-4608-8faf-74d05fdaa7ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:53.758012 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da12d1-e060-4608-8faf-74d05fdaa7ca admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:53.762495 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d1d82-0ac4-40a6-abef-472da7f3a984 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:53.763237 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d1d82-0ac4-40a6-abef-472da7f3a984 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:53.763708 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d1d82-0ac4-40a6-abef-472da7f3a984 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:53.764060 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d1d82-0ac4-40a6-abef-472da7f3a984 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:53.778727 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d1d82-0ac4-40a6-abef-472da7f3a984 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:53.781095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d1d82-0ac4-40a6-abef-472da7f3a984 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:53.797005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1029/2075] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:53.927164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d27964-8ff1-4b93-bc41-126342066e94 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:53.928645 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-26d27964-8ff1-4b93-bc41-126342066e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:53.942244 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d27964-8ff1-4b93-bc41-126342066e94 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:53.955192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848677b9-e187-43ca-8ed2-81fbc8284a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:53.955944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848677b9-e187-43ca-8ed2-81fbc8284a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:53.956227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848677b9-e187-43ca-8ed2-81fbc8284a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:53.956500 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848677b9-e187-43ca-8ed2-81fbc8284a99 admin admin] PATH_INFO: `/v3/projects/b3367b2540374bc0867934cc95856d0b` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:53.957833 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848677b9-e187-43ca-8ed2-81fbc8284a99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3367b2540374bc0867934cc95856d0b)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:53.959810 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848677b9-e187-43ca-8ed2-81fbc8284a99 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:54.000255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:54.023284 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-848677b9-e187-43ca-8ed2-81fbc8284a99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3367b2540374bc0867934cc95856d0b', 'request_id': 'req-848677b9-e187-43ca-8ed2-81fbc8284a99'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:54.033107 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-848677b9-e187-43ca-8ed2-81fbc8284a99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3367b2540374bc0867934cc95856d0b', 'request_id': 'req-848677b9-e187-43ca-8ed2-81fbc8284a99'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:54.040030 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-848677b9-e187-43ca-8ed2-81fbc8284a99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3367b2540374bc0867934cc95856d0b', 'request_id': 'req-848677b9-e187-43ca-8ed2-81fbc8284a99'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:54.061221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-848677b9-e187-43ca-8ed2-81fbc8284a99 admin admin] The token cache is being invalidate because project b3367b2540374bc0867934cc95856d0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:54.061692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-848677b9-e187-43ca-8ed2-81fbc8284a99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:54.062456 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-848677b9-e187-43ca-8ed2-81fbc8284a99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:54.064264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1047/2076] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:53 2022] DELETE /identity/v3/projects/b3367b2540374bc0867934cc95856d0b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:54.069648 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c91af3-2e47-41f2-8536-8f98cc433fbd None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:54.070214 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c91af3-2e47-41f2-8536-8f98cc433fbd None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:54.070214 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c91af3-2e47-41f2-8536-8f98cc433fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:54.091692 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-58c91af3-2e47-41f2-8536-8f98cc433fbd None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:54.092918 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-58c91af3-2e47-41f2-8536-8f98cc433fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:54.157771 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848677b9-e187-43ca-8ed2-81fbc8284a99 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:54.159194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-848677b9-e187-43ca-8ed2-81fbc8284a99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:54.165998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1030/2077] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:54.175198 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebc36a-7e84-46be-b086-9da54fe431b5 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:54.176292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebc36a-7e84-46be-b086-9da54fe431b5 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:54.176292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebc36a-7e84-46be-b086-9da54fe431b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:54.180783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848677b9-e187-43ca-8ed2-81fbc8284a99 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:54.189448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb5990-357f-4713-99e9-9948a7e9bbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:54.190319 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb5990-357f-4713-99e9-9948a7e9bbcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:54.190573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb5990-357f-4713-99e9-9948a7e9bbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:54.190963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb5990-357f-4713-99e9-9948a7e9bbcd admin admin] PATH_INFO: `/v3/projects/0c542fb4e5634418bcf3070f60c98fdb` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:54.193327 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb5990-357f-4713-99e9-9948a7e9bbcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c542fb4e5634418bcf3070f60c98fdb)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:54.195585 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb5990-357f-4713-99e9-9948a7e9bbcd admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:54.199457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-65ebc36a-7e84-46be-b086-9da54fe431b5 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:54.200964 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-65ebc36a-7e84-46be-b086-9da54fe431b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:54.209736 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1031/2078] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:54.213162 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebc36a-7e84-46be-b086-9da54fe431b5 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:54.223158 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebc36a-7e84-46be-b086-9da54fe431b5 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:54.228103 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aeb18c-08ad-40ec-8857-88751ce98341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:54.228671 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aeb18c-08ad-40ec-8857-88751ce98341 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:54.229169 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aeb18c-08ad-40ec-8857-88751ce98341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:54.229435 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aeb18c-08ad-40ec-8857-88751ce98341 admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:54.230205 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aeb18c-08ad-40ec-8857-88751ce98341 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:39:54.230734 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aeb18c-08ad-40ec-8857-88751ce98341 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:54.232273 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aeb18c-08ad-40ec-8857-88751ce98341 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:54.244789 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1032/2079] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:39:54.249084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aeb18c-08ad-40ec-8857-88751ce98341 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:54.250556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4afb5990-357f-4713-99e9-9948a7e9bbcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c542fb4e5634418bcf3070f60c98fdb', 'request_id': 'req-4afb5990-357f-4713-99e9-9948a7e9bbcd'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:54.259468 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aeb18c-08ad-40ec-8857-88751ce98341 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:54.260951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4afb5990-357f-4713-99e9-9948a7e9bbcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c542fb4e5634418bcf3070f60c98fdb', 'request_id': 'req-4afb5990-357f-4713-99e9-9948a7e9bbcd'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:54.268918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4afb5990-357f-4713-99e9-9948a7e9bbcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c542fb4e5634418bcf3070f60c98fdb', 'request_id': 'req-4afb5990-357f-4713-99e9-9948a7e9bbcd'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:54.275767 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:54.282691 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4afb5990-357f-4713-99e9-9948a7e9bbcd admin admin] The token cache is being invalidate because project 0c542fb4e5634418bcf3070f60c98fdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:54.282979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4afb5990-357f-4713-99e9-9948a7e9bbcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:54.283663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4afb5990-357f-4713-99e9-9948a7e9bbcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:54.287310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1048/2080] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:54 2022] DELETE /identity/v3/projects/0c542fb4e5634418bcf3070f60c98fdb => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:39:54.290876 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb5990-357f-4713-99e9-9948a7e9bbcd admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:54.292738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4afb5990-357f-4713-99e9-9948a7e9bbcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:54.337821 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37415134-7496-4a78-92c9-03b5e99a2a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:54.338445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37415134-7496-4a78-92c9-03b5e99a2a9d admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:54.338757 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37415134-7496-4a78-92c9-03b5e99a2a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:54.339055 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37415134-7496-4a78-92c9-03b5e99a2a9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:54.340370 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37415134-7496-4a78-92c9-03b5e99a2a9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:54.342310 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37415134-7496-4a78-92c9-03b5e99a2a9d admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:54.365949 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb5990-357f-4713-99e9-9948a7e9bbcd admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:54.371732 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455cb3e-2c90-4d12-85d2-551e82816428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:54.372600 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455cb3e-2c90-4d12-85d2-551e82816428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:54.372871 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455cb3e-2c90-4d12-85d2-551e82816428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:54.373146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455cb3e-2c90-4d12-85d2-551e82816428 admin admin] PATH_INFO: `/v3/users/98c5f8909de241ecae28fa43a7ec27c5` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:54.375131 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455cb3e-2c90-4d12-85d2-551e82816428 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98c5f8909de241ecae28fa43a7ec27c5)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:54.379421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1033/2081] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:54 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 04:39:54.379622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455cb3e-2c90-4d12-85d2-551e82816428 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:54.380872 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37415134-7496-4a78-92c9-03b5e99a2a9d admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:54.382849 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-37415134-7496-4a78-92c9-03b5e99a2a9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:54.423404 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:54.429719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5455cb3e-2c90-4d12-85d2-551e82816428 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98c5f8909de241ecae28fa43a7ec27c5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:54.451684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5455cb3e-2c90-4d12-85d2-551e82816428 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98c5f8909de241ecae28fa43a7ec27c5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:54.452121 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5455cb3e-2c90-4d12-85d2-551e82816428 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98c5f8909de241ecae28fa43a7ec27c5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:54.466566 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5455cb3e-2c90-4d12-85d2-551e82816428 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98c5f8909de241ecae28fa43a7ec27c5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:54.467307 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5455cb3e-2c90-4d12-85d2-551e82816428 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98c5f8909de241ecae28fa43a7ec27c5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:54.473316 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37415134-7496-4a78-92c9-03b5e99a2a9d admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:54.481179 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e567aa0-4ce3-4637-85f2-95ff5a6683ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:54.482340 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e567aa0-4ce3-4637-85f2-95ff5a6683ac admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:54.482846 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e567aa0-4ce3-4637-85f2-95ff5a6683ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:54.483356 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e567aa0-4ce3-4637-85f2-95ff5a6683ac admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:54.484262 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e567aa0-4ce3-4637-85f2-95ff5a6683ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:54.486507 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1049/2082] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:54 2022] DELETE /identity/v3/users/98c5f8909de241ecae28fa43a7ec27c5 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:39:54.487614 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e567aa0-4ce3-4637-85f2-95ff5a6683ac admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:54.528922 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1034/2083] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:54 2022] POST /identity/v3/users => generated 287 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:54.532352 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455cb3e-2c90-4d12-85d2-551e82816428 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:54.534462 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-5455cb3e-2c90-4d12-85d2-551e82816428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:54.618365 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455cb3e-2c90-4d12-85d2-551e82816428 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:54.625595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702b9a8-c3c5-48ab-af6c-fa21fb83d401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:54.626016 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702b9a8-c3c5-48ab-af6c-fa21fb83d401 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:54.626336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702b9a8-c3c5-48ab-af6c-fa21fb83d401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:54.626741 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702b9a8-c3c5-48ab-af6c-fa21fb83d401 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:54.627462 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d702b9a8-c3c5-48ab-af6c-fa21fb83d401 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:54.629077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d702b9a8-c3c5-48ab-af6c-fa21fb83d401 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:54.640790 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1050/2084] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:54.644920 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e567aa0-4ce3-4637-85f2-95ff5a6683ac admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:54.656265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e567aa0-4ce3-4637-85f2-95ff5a6683ac admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:54.661152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f1960-3abd-4c97-b32a-191dc06acd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:54.661814 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f1960-3abd-4c97-b32a-191dc06acd48 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:54.662767 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f1960-3abd-4c97-b32a-191dc06acd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:54.662767 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f1960-3abd-4c97-b32a-191dc06acd48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:54.663051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247f1960-3abd-4c97-b32a-191dc06acd48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:54.664654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247f1960-3abd-4c97-b32a-191dc06acd48 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:54.677852 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1035/2085] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:54.682596 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702b9a8-c3c5-48ab-af6c-fa21fb83d401 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:54.693110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702b9a8-c3c5-48ab-af6c-fa21fb83d401 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:54.698423 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1ed91-0073-4c9b-9af7-cb11566768ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:54.699716 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1ed91-0073-4c9b-9af7-cb11566768ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:54.700180 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1ed91-0073-4c9b-9af7-cb11566768ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:54.700458 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1ed91-0073-4c9b-9af7-cb11566768ae admin admin] PATH_INFO: `/v3/projects/8ccec168634142b69f1cac5ff04f9657/users/0834d7f82b12405fb591dca9af25b884/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:54.725905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1ed91-0073-4c9b-9af7-cb11566768ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ccec168634142b69f1cac5ff04f9657, user_id=0834d7f82b12405fb591dca9af25b884, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:54.728675 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1ed91-0073-4c9b-9af7-cb11566768ae admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:54.744063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1051/2086] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:54 2022] PUT /identity/v3/projects/8ccec168634142b69f1cac5ff04f9657/users/0834d7f82b12405fb591dca9af25b884/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:54.754494 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d764b-0520-4317-b2cc-3e042e1a4dc4 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:54.754860 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d764b-0520-4317-b2cc-3e042e1a4dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:54.755201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d764b-0520-4317-b2cc-3e042e1a4dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:54.787466 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-a70d764b-0520-4317-b2cc-3e042e1a4dc4 None None] MFA Rules not processed for user `0834d7f82b12405fb591dca9af25b884`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:54.789625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-a70d764b-0520-4317-b2cc-3e042e1a4dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:54.873326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1036/2087] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:54.883001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1ed91-0073-4c9b-9af7-cb11566768ae admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:54.885229 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b0c1ed91-0073-4c9b-9af7-cb11566768ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:54.969517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1ed91-0073-4c9b-9af7-cb11566768ae admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:54.974845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68f5c3-cf36-416b-adaa-38188ec8d099 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:54.975709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68f5c3-cf36-416b-adaa-38188ec8d099 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:54.976130 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68f5c3-cf36-416b-adaa-38188ec8d099 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:54.976510 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68f5c3-cf36-416b-adaa-38188ec8d099 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:54.987173 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f68f5c3-cf36-416b-adaa-38188ec8d099 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:54.988626 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f68f5c3-cf36-416b-adaa-38188ec8d099 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:54.999147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1052/2088] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:55.025178 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d764b-0520-4317-b2cc-3e042e1a4dc4 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:55.054802 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:55.088661 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68f5c3-cf36-416b-adaa-38188ec8d099 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:55.092386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9f68f5c3-cf36-416b-adaa-38188ec8d099 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:55.179642 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d764b-0520-4317-b2cc-3e042e1a4dc4 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:55.181184 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:55.187707 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13b3b9-a2e2-41bf-b59f-9d6cc93039fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:55.188594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13b3b9-a2e2-41bf-b59f-9d6cc93039fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:55.189037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13b3b9-a2e2-41bf-b59f-9d6cc93039fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:55.189467 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13b3b9-a2e2-41bf-b59f-9d6cc93039fb admin admin] PATH_INFO: `/v3/users/0834d7f82b12405fb591dca9af25b884` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:55.190815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13b3b9-a2e2-41bf-b59f-9d6cc93039fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0834d7f82b12405fb591dca9af25b884)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:55.193562 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13b3b9-a2e2-41bf-b59f-9d6cc93039fb admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:55.239488 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68f5c3-cf36-416b-adaa-38188ec8d099 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:55.247517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd0dec-8e1a-4d5c-aabd-51b5f6f36b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:55.248229 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd0dec-8e1a-4d5c-aabd-51b5f6f36b68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:55.248580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd0dec-8e1a-4d5c-aabd-51b5f6f36b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:55.248878 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd0dec-8e1a-4d5c-aabd-51b5f6f36b68 admin admin] PATH_INFO: `/v3/projects/e0b51e63301048fab44ee19fbd823261` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:55.251862 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd0dec-8e1a-4d5c-aabd-51b5f6f36b68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0b51e63301048fab44ee19fbd823261)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:55.255231 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd0dec-8e1a-4d5c-aabd-51b5f6f36b68 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:55.263337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-7e13b3b9-a2e2-41bf-b59f-9d6cc93039fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0834d7f82b12405fb591dca9af25b884'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:55.264427 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-7e13b3b9-a2e2-41bf-b59f-9d6cc93039fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0834d7f82b12405fb591dca9af25b884'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:55.285479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-7e13b3b9-a2e2-41bf-b59f-9d6cc93039fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0834d7f82b12405fb591dca9af25b884'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:55.300334 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-7e13b3b9-a2e2-41bf-b59f-9d6cc93039fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0834d7f82b12405fb591dca9af25b884'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:55.302982 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-7e13b3b9-a2e2-41bf-b59f-9d6cc93039fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0834d7f82b12405fb591dca9af25b884'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:55.332222 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1037/2089] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:55 2022] DELETE /identity/v3/users/0834d7f82b12405fb591dca9af25b884 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:55.338874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8aaee7-d9b4-4c1e-b941-d8638f004c27 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:55.339409 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8aaee7-d9b4-4c1e-b941-d8638f004c27 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:55.340587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8aaee7-d9b4-4c1e-b941-d8638f004c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:55.358139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-badd0dec-8e1a-4d5c-aabd-51b5f6f36b68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0b51e63301048fab44ee19fbd823261', 'request_id': 'req-badd0dec-8e1a-4d5c-aabd-51b5f6f36b68'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:55.372228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-badd0dec-8e1a-4d5c-aabd-51b5f6f36b68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0b51e63301048fab44ee19fbd823261', 'request_id': 'req-badd0dec-8e1a-4d5c-aabd-51b5f6f36b68'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:55.376939 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-5d8aaee7-d9b4-4c1e-b941-d8638f004c27 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:55.378259 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5d8aaee7-d9b4-4c1e-b941-d8638f004c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:55.383388 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-badd0dec-8e1a-4d5c-aabd-51b5f6f36b68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0b51e63301048fab44ee19fbd823261', 'request_id': 'req-badd0dec-8e1a-4d5c-aabd-51b5f6f36b68'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:55.405281 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-badd0dec-8e1a-4d5c-aabd-51b5f6f36b68 admin admin] The token cache is being invalidate because project e0b51e63301048fab44ee19fbd823261 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:55.405606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-badd0dec-8e1a-4d5c-aabd-51b5f6f36b68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:55.406304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-badd0dec-8e1a-4d5c-aabd-51b5f6f36b68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:55.408487 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1053/2090] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:55 2022] DELETE /identity/v3/projects/e0b51e63301048fab44ee19fbd823261 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:55.414306 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac9ce8-8b14-4ee6-88eb-ae979f6166e6 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:55.414506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac9ce8-8b14-4ee6-88eb-ae979f6166e6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:55.414753 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac9ce8-8b14-4ee6-88eb-ae979f6166e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:55.463504 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-34ac9ce8-8b14-4ee6-88eb-ae979f6166e6 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:55.466419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-34ac9ce8-8b14-4ee6-88eb-ae979f6166e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:55.472560 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1038/2091] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:55.481444 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1054/2092] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:55.487763 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8aaee7-d9b4-4c1e-b941-d8638f004c27 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:55.489907 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb3733-6c69-49c9-b4ec-c3e5eabf9c17 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:55.490194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb3733-6c69-49c9-b4ec-c3e5eabf9c17 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:55.490474 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb3733-6c69-49c9-b4ec-c3e5eabf9c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:55.493326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5d8aaee7-d9b4-4c1e-b941-d8638f004c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:55.537013 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-77eb3733-6c69-49c9-b4ec-c3e5eabf9c17 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:55.540057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-77eb3733-6c69-49c9-b4ec-c3e5eabf9c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:55.555331 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1055/2093] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:55.556578 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:55.556804 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb3733-6c69-49c9-b4ec-c3e5eabf9c17 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:55.568552 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb3733-6c69-49c9-b4ec-c3e5eabf9c17 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:55.576936 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b41265-8696-4ee3-a88b-2bcaed561bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:55.577886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b41265-8696-4ee3-a88b-2bcaed561bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:55.578358 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b41265-8696-4ee3-a88b-2bcaed561bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:55.578783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b41265-8696-4ee3-a88b-2bcaed561bd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:55.579629 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b41265-8696-4ee3-a88b-2bcaed561bd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:39:55.580081 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b41265-8696-4ee3-a88b-2bcaed561bd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:55.582190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b41265-8696-4ee3-a88b-2bcaed561bd7 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:55.597734 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1056/2094] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:39:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:39:55.602867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b41265-8696-4ee3-a88b-2bcaed561bd7 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:55.616672 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8aaee7-d9b4-4c1e-b941-d8638f004c27 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:55.616826 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b41265-8696-4ee3-a88b-2bcaed561bd7 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:55.623829 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a1b77-4420-412d-8877-4ef2884b56ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:55.624512 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81df39-fa49-4836-a25b-7f0dab19ab5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:55.625235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81df39-fa49-4836-a25b-7f0dab19ab5c service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:55.625619 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81df39-fa49-4836-a25b-7f0dab19ab5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:55.625962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81df39-fa49-4836-a25b-7f0dab19ab5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:55.626222 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a1b77-4420-412d-8877-4ef2884b56ab admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:55.626766 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a1b77-4420-412d-8877-4ef2884b56ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:55.627959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a1b77-4420-412d-8877-4ef2884b56ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:55.632070 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19a1b77-4420-412d-8877-4ef2884b56ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:55.635204 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19a1b77-4420-412d-8877-4ef2884b56ab admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:55.649168 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c81df39-fa49-4836-a25b-7f0dab19ab5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:55.652112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c81df39-fa49-4836-a25b-7f0dab19ab5c service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:55.682476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1039/2095] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:55.706952 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1057/2096] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:55 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:55.712293 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81df39-fa49-4836-a25b-7f0dab19ab5c service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:55.736457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81df39-fa49-4836-a25b-7f0dab19ab5c service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:55.922450 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ba7dc-626f-4779-b06e-a2e0b95998e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:55.923562 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ba7dc-626f-4779-b06e-a2e0b95998e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:55.923562 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ba7dc-626f-4779-b06e-a2e0b95998e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:55.923783 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ba7dc-626f-4779-b06e-a2e0b95998e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:55.925023 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ba7dc-626f-4779-b06e-a2e0b95998e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:55.927613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ba7dc-626f-4779-b06e-a2e0b95998e3 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:55.962490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1040/2097] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:55 2022] POST /identity/v3/users => generated 293 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:55.966514 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a1b77-4420-412d-8877-4ef2884b56ab admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:55.981618 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a1b77-4420-412d-8877-4ef2884b56ab admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:55.994440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e3307-93b8-4127-936c-38efa71d378d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:55.995241 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e3307-93b8-4127-936c-38efa71d378d admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:55.995527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e3307-93b8-4127-936c-38efa71d378d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:55.996431 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e3307-93b8-4127-936c-38efa71d378d admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:55.997593 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e3307-93b8-4127-936c-38efa71d378d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:55.999212 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e3307-93b8-4127-936c-38efa71d378d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:56.035842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1058/2098] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:56.036814 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ba7dc-626f-4779-b06e-a2e0b95998e3 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:56.050623 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ba7dc-626f-4779-b06e-a2e0b95998e3 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:56.058777 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624878e0-9f52-40fa-b931-a319cba6a300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:56.059467 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624878e0-9f52-40fa-b931-a319cba6a300 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:56.060138 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624878e0-9f52-40fa-b931-a319cba6a300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:56.060138 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624878e0-9f52-40fa-b931-a319cba6a300 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:56.060662 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624878e0-9f52-40fa-b931-a319cba6a300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:56.064808 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624878e0-9f52-40fa-b931-a319cba6a300 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:56.085536 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1041/2099] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:56.090086 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e3307-93b8-4127-936c-38efa71d378d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:56.123528 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e3307-93b8-4127-936c-38efa71d378d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:56.130552 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c62f8-5c22-45f8-81a7-fed6a53295d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:56.131277 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c62f8-5c22-45f8-81a7-fed6a53295d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:56.131811 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c62f8-5c22-45f8-81a7-fed6a53295d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:56.132114 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c62f8-5c22-45f8-81a7-fed6a53295d1 admin admin] PATH_INFO: `/v3/projects/b5041c381beb4768937bee8afba7d5c8/users/acb5dc2aa264403b88c85b874adbff2a/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:56.190718 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658c62f8-5c22-45f8-81a7-fed6a53295d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5041c381beb4768937bee8afba7d5c8, user_id=acb5dc2aa264403b88c85b874adbff2a, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:56.193751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658c62f8-5c22-45f8-81a7-fed6a53295d1 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:56.236483 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80844a5e-ef90-4e55-98f1-6469fdc7b5d4 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:56.236677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80844a5e-ef90-4e55-98f1-6469fdc7b5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:56.236920 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80844a5e-ef90-4e55-98f1-6469fdc7b5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:56.243211 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1059/2100] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:56 2022] PUT /identity/v3/projects/b5041c381beb4768937bee8afba7d5c8/users/acb5dc2aa264403b88c85b874adbff2a/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:56.288354 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-80844a5e-ef90-4e55-98f1-6469fdc7b5d4 None None] MFA Rules not processed for user `acb5dc2aa264403b88c85b874adbff2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:56.290043 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-80844a5e-ef90-4e55-98f1-6469fdc7b5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:56.400655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1042/2101] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:56.415621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80844a5e-ef90-4e55-98f1-6469fdc7b5d4 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:56.416525 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c62f8-5c22-45f8-81a7-fed6a53295d1 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:56.417127 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-80844a5e-ef90-4e55-98f1-6469fdc7b5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:56.420460 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-658c62f8-5c22-45f8-81a7-fed6a53295d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:56.498933 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:56.531079 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:56.552938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80844a5e-ef90-4e55-98f1-6469fdc7b5d4 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:56.567524 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c62f8-5c22-45f8-81a7-fed6a53295d1 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:56.568278 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88fc9c-5fe1-449d-896f-113027ac62eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:56.569035 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88fc9c-5fe1-449d-896f-113027ac62eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:56.571175 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88fc9c-5fe1-449d-896f-113027ac62eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:56.571937 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88fc9c-5fe1-449d-896f-113027ac62eb admin admin] PATH_INFO: `/v3/users/b05037107965471bbffe9be332758fc0` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:56.576626 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88fc9c-5fe1-449d-896f-113027ac62eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b05037107965471bbffe9be332758fc0)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:56.579091 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7482c-5589-4d3c-b9b9-a6a9293d12d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:56.581045 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88fc9c-5fe1-449d-896f-113027ac62eb admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:56.582439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7482c-5589-4d3c-b9b9-a6a9293d12d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:56.582439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7482c-5589-4d3c-b9b9-a6a9293d12d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:56.582439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7482c-5589-4d3c-b9b9-a6a9293d12d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:56.597870 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf7482c-5589-4d3c-b9b9-a6a9293d12d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:56.600416 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf7482c-5589-4d3c-b9b9-a6a9293d12d3 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:56.611517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1060/2102] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:56.637340 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7482c-5589-4d3c-b9b9-a6a9293d12d3 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:56.651716 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9d88fc9c-5fe1-449d-896f-113027ac62eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b05037107965471bbffe9be332758fc0'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:56.653773 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9d88fc9c-5fe1-449d-896f-113027ac62eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b05037107965471bbffe9be332758fc0'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:56.654438 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7482c-5589-4d3c-b9b9-a6a9293d12d3 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:56.661285 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f00146-0ad3-43d1-8aa0-188cad93426c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:56.662309 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f00146-0ad3-43d1-8aa0-188cad93426c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:56.662751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f00146-0ad3-43d1-8aa0-188cad93426c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:56.663182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f00146-0ad3-43d1-8aa0-188cad93426c admin admin] PATH_INFO: `/v3/users/acb5dc2aa264403b88c85b874adbff2a` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:56.665525 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f00146-0ad3-43d1-8aa0-188cad93426c admin admin] RBAC: Authorizing `identity:delete_user(user_id=acb5dc2aa264403b88c85b874adbff2a)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:56.668127 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f00146-0ad3-43d1-8aa0-188cad93426c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:56.669870 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9d88fc9c-5fe1-449d-896f-113027ac62eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b05037107965471bbffe9be332758fc0'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:56.685790 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9d88fc9c-5fe1-449d-896f-113027ac62eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b05037107965471bbffe9be332758fc0'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:56.686639 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9d88fc9c-5fe1-449d-896f-113027ac62eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b05037107965471bbffe9be332758fc0'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:56.717478 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1043/2103] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:56 2022] DELETE /identity/v3/users/b05037107965471bbffe9be332758fc0 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:39:56.719636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34690889-a997-497f-a283-75f1cb16d059 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:56.719636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34690889-a997-497f-a283-75f1cb16d059 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:56.719848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34690889-a997-497f-a283-75f1cb16d059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:56.751975 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-34690889-a997-497f-a283-75f1cb16d059 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:56.753242 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-34690889-a997-497f-a283-75f1cb16d059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:56.785164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-56f00146-0ad3-43d1-8aa0-188cad93426c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acb5dc2aa264403b88c85b874adbff2a'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:56.795796 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-56f00146-0ad3-43d1-8aa0-188cad93426c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acb5dc2aa264403b88c85b874adbff2a'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:56.796806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-56f00146-0ad3-43d1-8aa0-188cad93426c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acb5dc2aa264403b88c85b874adbff2a'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:56.821006 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-56f00146-0ad3-43d1-8aa0-188cad93426c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acb5dc2aa264403b88c85b874adbff2a'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:56.821894 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-56f00146-0ad3-43d1-8aa0-188cad93426c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acb5dc2aa264403b88c85b874adbff2a'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:56.843678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1061/2104] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:56 2022] DELETE /identity/v3/users/acb5dc2aa264403b88c85b874adbff2a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:56.848493 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ed375-b08f-4021-8868-a4a6606b3d58 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:56.848613 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ed375-b08f-4021-8868-a4a6606b3d58 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:56.848897 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ed375-b08f-4021-8868-a4a6606b3d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:56.858938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1044/2105] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:56.872826 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34690889-a997-497f-a283-75f1cb16d059 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:56.874648 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-34690889-a997-497f-a283-75f1cb16d059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:56.884343 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-fb7ed375-b08f-4021-8868-a4a6606b3d58 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:56.892136 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fb7ed375-b08f-4021-8868-a4a6606b3d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:56.990772 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34690889-a997-497f-a283-75f1cb16d059 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:57.010561 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:57.044740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1062/2106] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:57.060083 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ed375-b08f-4021-8868-a4a6606b3d58 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:57.155320 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d45d42-d2fe-465c-8b40-cd7a61359b26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:57.156071 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d45d42-d2fe-465c-8b40-cd7a61359b26 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:57.157208 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d45d42-d2fe-465c-8b40-cd7a61359b26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:57.157588 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d45d42-d2fe-465c-8b40-cd7a61359b26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:57.177052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d45d42-d2fe-465c-8b40-cd7a61359b26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:57.178654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d45d42-d2fe-465c-8b40-cd7a61359b26 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:57.181273 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ed375-b08f-4021-8868-a4a6606b3d58 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:57.187430 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ec5d5-3fc9-4fc9-be78-d515e1d72035 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:57.190524 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ec5d5-3fc9-4fc9-be78-d515e1d72035 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:57.191539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ec5d5-3fc9-4fc9-be78-d515e1d72035 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:57.191636 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ec5d5-3fc9-4fc9-be78-d515e1d72035 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:57.195190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1045/2107] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:57.196256 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d45d42-d2fe-465c-8b40-cd7a61359b26 service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:57.197652 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d8d45d42-d2fe-465c-8b40-cd7a61359b26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:57.209080 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ec5d5-3fc9-4fc9-be78-d515e1d72035 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:57.210924 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ec5d5-3fc9-4fc9-be78-d515e1d72035 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:57.216613 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:57.217521 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d45d42-d2fe-465c-8b40-cd7a61359b26 service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:57.224594 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1063/2108] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:39:57.225736 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f63aa0-7846-43ba-bf76-432889499d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:57.226292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f63aa0-7846-43ba-bf76-432889499d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:57.226798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f63aa0-7846-43ba-bf76-432889499d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:57.227165 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f63aa0-7846-43ba-bf76-432889499d1f admin admin] PATH_INFO: `/v3/projects/8ccec168634142b69f1cac5ff04f9657` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:57.228647 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f63aa0-7846-43ba-bf76-432889499d1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ccec168634142b69f1cac5ff04f9657)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:57.233438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f63aa0-7846-43ba-bf76-432889499d1f admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:57.310087 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-e7f63aa0-7846-43ba-bf76-432889499d1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ccec168634142b69f1cac5ff04f9657', 'request_id': 'req-e7f63aa0-7846-43ba-bf76-432889499d1f'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:57.334047 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-e7f63aa0-7846-43ba-bf76-432889499d1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ccec168634142b69f1cac5ff04f9657', 'request_id': 'req-e7f63aa0-7846-43ba-bf76-432889499d1f'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:57.348326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-e7f63aa0-7846-43ba-bf76-432889499d1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ccec168634142b69f1cac5ff04f9657', 'request_id': 'req-e7f63aa0-7846-43ba-bf76-432889499d1f'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:57.372232 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-e7f63aa0-7846-43ba-bf76-432889499d1f admin admin] The token cache is being invalidate because project 8ccec168634142b69f1cac5ff04f9657 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:57.372672 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-e7f63aa0-7846-43ba-bf76-432889499d1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:57.374611 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-e7f63aa0-7846-43ba-bf76-432889499d1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:57.383694 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440e6c1-6967-4c63-b2cb-99c94f3c3046 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:57.384059 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440e6c1-6967-4c63-b2cb-99c94f3c3046 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:57.384328 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440e6c1-6967-4c63-b2cb-99c94f3c3046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:57.388457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1046/2109] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:57 2022] DELETE /identity/v3/projects/8ccec168634142b69f1cac5ff04f9657 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:57.416002 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-2440e6c1-6967-4c63-b2cb-99c94f3c3046 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:57.417241 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2440e6c1-6967-4c63-b2cb-99c94f3c3046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:57.506705 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1064/2110] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:57.516748 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7607789-3358-4a95-bb5f-040796994b0a None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:57.517230 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7607789-3358-4a95-bb5f-040796994b0a None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:57.517339 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7607789-3358-4a95-bb5f-040796994b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:57.540709 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-d7607789-3358-4a95-bb5f-040796994b0a None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:57.542043 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d7607789-3358-4a95-bb5f-040796994b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:57.556209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440e6c1-6967-4c63-b2cb-99c94f3c3046 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:57.562145 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1047/2111] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:39:57.568359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440e6c1-6967-4c63-b2cb-99c94f3c3046 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:57.573905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff7435-09c7-428f-b0e9-6b70630229d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:57.574831 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff7435-09c7-428f-b0e9-6b70630229d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:57.575139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff7435-09c7-428f-b0e9-6b70630229d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:57.575403 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff7435-09c7-428f-b0e9-6b70630229d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:57.576123 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff7435-09c7-428f-b0e9-6b70630229d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:39:57.576432 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff7435-09c7-428f-b0e9-6b70630229d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:57.578091 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff7435-09c7-428f-b0e9-6b70630229d2 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:57.595408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1065/2112] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:39:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:39:57.600018 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7607789-3358-4a95-bb5f-040796994b0a None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:57.616303 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7607789-3358-4a95-bb5f-040796994b0a None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:57.623693 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ccf391-deed-4984-93bc-32574ddde4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:57.624518 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccf391-deed-4984-93bc-32574ddde4d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:57.624803 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccf391-deed-4984-93bc-32574ddde4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:57.625075 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccf391-deed-4984-93bc-32574ddde4d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:57.625792 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ccf391-deed-4984-93bc-32574ddde4d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:57.627762 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ccf391-deed-4984-93bc-32574ddde4d0 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:57.668546 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1048/2113] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:57 2022] POST /identity/v3/projects => generated 371 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:57.674656 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff7435-09c7-428f-b0e9-6b70630229d2 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:57.687928 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff7435-09c7-428f-b0e9-6b70630229d2 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:57.787359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ea5d1-8b13-4578-83c7-0320763b314b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:57.787984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ea5d1-8b13-4578-83c7-0320763b314b admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:57.788548 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ea5d1-8b13-4578-83c7-0320763b314b admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:57.788842 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ea5d1-8b13-4578-83c7-0320763b314b admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:57.789696 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ea5d1-8b13-4578-83c7-0320763b314b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:57.792299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ea5d1-8b13-4578-83c7-0320763b314b admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:57.820051 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1066/2114] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:57 2022] POST /identity/v3/users => generated 286 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:57.823574 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ccf391-deed-4984-93bc-32574ddde4d0 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:57.838781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ccf391-deed-4984-93bc-32574ddde4d0 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:57.844883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d917d6a-c668-4b5f-8384-6fe231b67e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:57.845631 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d917d6a-c668-4b5f-8384-6fe231b67e01 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:57.845990 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d917d6a-c668-4b5f-8384-6fe231b67e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:57.846347 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d917d6a-c668-4b5f-8384-6fe231b67e01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:57.847643 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d917d6a-c668-4b5f-8384-6fe231b67e01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:57.850124 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d917d6a-c668-4b5f-8384-6fe231b67e01 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:57.863557 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1049/2115] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 04:39:57.867390 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ea5d1-8b13-4578-83c7-0320763b314b admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:57.879444 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ea5d1-8b13-4578-83c7-0320763b314b admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:57.884563 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8688de3-5a20-449b-a31d-8e6b301dc542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:57.885394 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8688de3-5a20-449b-a31d-8e6b301dc542 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:57.885812 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8688de3-5a20-449b-a31d-8e6b301dc542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:57.886849 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8688de3-5a20-449b-a31d-8e6b301dc542 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:57.887986 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8688de3-5a20-449b-a31d-8e6b301dc542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:57.899552 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8688de3-5a20-449b-a31d-8e6b301dc542 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:57.924078 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1067/2116] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:57.929719 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d917d6a-c668-4b5f-8384-6fe231b67e01 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:57.956684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d917d6a-c668-4b5f-8384-6fe231b67e01 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:57.972896 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8688de3-5a20-449b-a31d-8e6b301dc542 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:57.973647 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052beb65-1a03-40a1-9d93-bd2eaa975dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:57.976046 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052beb65-1a03-40a1-9d93-bd2eaa975dfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:57.977678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b8688de3-5a20-449b-a31d-8e6b301dc542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:57.977820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052beb65-1a03-40a1-9d93-bd2eaa975dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:57.978308 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052beb65-1a03-40a1-9d93-bd2eaa975dfa admin admin] PATH_INFO: `/v3/projects/23ec86581683456b8c52c649eed292e7/users/1b23dd1311554910a19399aa74df9e4c/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:58.013507 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052beb65-1a03-40a1-9d93-bd2eaa975dfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=23ec86581683456b8c52c649eed292e7, user_id=1b23dd1311554910a19399aa74df9e4c, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:58.013948 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8688de3-5a20-449b-a31d-8e6b301dc542 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:58.017026 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052beb65-1a03-40a1-9d93-bd2eaa975dfa admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:58.020953 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ed300-2186-4d21-8031-6d479becea8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:58.021884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ed300-2186-4d21-8031-6d479becea8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:58.022871 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ed300-2186-4d21-8031-6d479becea8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:58.023956 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ed300-2186-4d21-8031-6d479becea8c admin admin] PATH_INFO: `/v3/projects/767d2090d469421baa485c5d19cd61dc` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:58.025819 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ed300-2186-4d21-8031-6d479becea8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=767d2090d469421baa485c5d19cd61dc)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:58.028597 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ed300-2186-4d21-8031-6d479becea8c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:58.043500 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1050/2117] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:57 2022] PUT /identity/v3/projects/23ec86581683456b8c52c649eed292e7/users/1b23dd1311554910a19399aa74df9e4c/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:58.045944 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf56db-6e3c-412f-81e5-57793610feea None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:58.046270 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf56db-6e3c-412f-81e5-57793610feea None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:58.046560 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf56db-6e3c-412f-81e5-57793610feea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:58.080536 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-bbdf56db-6e3c-412f-81e5-57793610feea None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:58.082100 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-bbdf56db-6e3c-412f-81e5-57793610feea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:58.096198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-022ed300-2186-4d21-8031-6d479becea8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '767d2090d469421baa485c5d19cd61dc', 'request_id': 'req-022ed300-2186-4d21-8031-6d479becea8c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:58.103899 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-022ed300-2186-4d21-8031-6d479becea8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '767d2090d469421baa485c5d19cd61dc', 'request_id': 'req-022ed300-2186-4d21-8031-6d479becea8c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:58.107083 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:58.111689 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-022ed300-2186-4d21-8031-6d479becea8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '767d2090d469421baa485c5d19cd61dc', 'request_id': 'req-022ed300-2186-4d21-8031-6d479becea8c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:58.121362 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:58.127143 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-022ed300-2186-4d21-8031-6d479becea8c admin admin] The token cache is being invalidate because project 767d2090d469421baa485c5d19cd61dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:58.127143 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-022ed300-2186-4d21-8031-6d479becea8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:58.127920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-022ed300-2186-4d21-8031-6d479becea8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:58.129813 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1068/2118] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:57 2022] DELETE /identity/v3/projects/767d2090d469421baa485c5d19cd61dc => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:39:58.130965 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ed300-2186-4d21-8031-6d479becea8c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:58.133009 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-022ed300-2186-4d21-8031-6d479becea8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:58.153338 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1051/2119] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:58.157262 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf56db-6e3c-412f-81e5-57793610feea None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:58.157262 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-bbdf56db-6e3c-412f-81e5-57793610feea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:58.168195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf56db-6e3c-412f-81e5-57793610feea None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:58.173968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c1d0c-d706-4e32-8df3-1907f52ba36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:58.174792 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c1d0c-d706-4e32-8df3-1907f52ba36e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:58.175218 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c1d0c-d706-4e32-8df3-1907f52ba36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:58.175655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c1d0c-d706-4e32-8df3-1907f52ba36e admin admin] PATH_INFO: `/v3/users/840adf1a45c241728428d1194f9a7b18` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:58.177195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c1d0c-d706-4e32-8df3-1907f52ba36e admin admin] RBAC: Authorizing `identity:delete_user(user_id=840adf1a45c241728428d1194f9a7b18)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:58.179769 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c1d0c-d706-4e32-8df3-1907f52ba36e admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:58.213666 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ed300-2186-4d21-8031-6d479becea8c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:58.219570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643ee5f-dc6a-4a4e-a2b4-a472824c720e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:58.220268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643ee5f-dc6a-4a4e-a2b4-a472824c720e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:58.220554 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643ee5f-dc6a-4a4e-a2b4-a472824c720e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:58.220826 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643ee5f-dc6a-4a4e-a2b4-a472824c720e admin admin] PATH_INFO: `/v3/projects/b5041c381beb4768937bee8afba7d5c8` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:58.222068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3643ee5f-dc6a-4a4e-a2b4-a472824c720e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5041c381beb4768937bee8afba7d5c8)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:58.223998 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3643ee5f-dc6a-4a4e-a2b4-a472824c720e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:58.236978 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-b61c1d0c-d706-4e32-8df3-1907f52ba36e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '840adf1a45c241728428d1194f9a7b18'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:58.237769 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-b61c1d0c-d706-4e32-8df3-1907f52ba36e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '840adf1a45c241728428d1194f9a7b18'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:58.252940 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-b61c1d0c-d706-4e32-8df3-1907f52ba36e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '840adf1a45c241728428d1194f9a7b18'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:58.260248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-b61c1d0c-d706-4e32-8df3-1907f52ba36e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '840adf1a45c241728428d1194f9a7b18'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:58.261003 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-b61c1d0c-d706-4e32-8df3-1907f52ba36e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '840adf1a45c241728428d1194f9a7b18'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:58.278566 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1052/2120] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:58 2022] DELETE /identity/v3/users/840adf1a45c241728428d1194f9a7b18 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:58.279561 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-3643ee5f-dc6a-4a4e-a2b4-a472824c720e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5041c381beb4768937bee8afba7d5c8', 'request_id': 'req-3643ee5f-dc6a-4a4e-a2b4-a472824c720e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:58.282455 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c1d0c-d706-4e32-8df3-1907f52ba36e admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:58.283859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-b61c1d0c-d706-4e32-8df3-1907f52ba36e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:58.291565 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-3643ee5f-dc6a-4a4e-a2b4-a472824c720e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5041c381beb4768937bee8afba7d5c8', 'request_id': 'req-3643ee5f-dc6a-4a4e-a2b4-a472824c720e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:58.310651 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-3643ee5f-dc6a-4a4e-a2b4-a472824c720e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5041c381beb4768937bee8afba7d5c8', 'request_id': 'req-3643ee5f-dc6a-4a4e-a2b4-a472824c720e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:58.332623 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-3643ee5f-dc6a-4a4e-a2b4-a472824c720e admin admin] The token cache is being invalidate because project b5041c381beb4768937bee8afba7d5c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:58.333095 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-3643ee5f-dc6a-4a4e-a2b4-a472824c720e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:58.334010 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-3643ee5f-dc6a-4a4e-a2b4-a472824c720e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:58.335741 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1069/2121] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:58 2022] DELETE /identity/v3/projects/b5041c381beb4768937bee8afba7d5c8 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:58.460690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c1d0c-d706-4e32-8df3-1907f52ba36e admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:58.469886 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be006b4f-c0b8-45af-8a25-1243eb405b5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:58.470698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be006b4f-c0b8-45af-8a25-1243eb405b5b service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:58.473184 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be006b4f-c0b8-45af-8a25-1243eb405b5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:58.473649 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be006b4f-c0b8-45af-8a25-1243eb405b5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:58.478704 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-be006b4f-c0b8-45af-8a25-1243eb405b5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:58.640312 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643ee5f-dc6a-4a4e-a2b4-a472824c720e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:58.643152 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3643ee5f-dc6a-4a4e-a2b4-a472824c720e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:58.648695 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be006b4f-c0b8-45af-8a25-1243eb405b5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:58.650472 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be006b4f-c0b8-45af-8a25-1243eb405b5b service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:58.664373 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1053/2122] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:58.761049 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643ee5f-dc6a-4a4e-a2b4-a472824c720e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:58.771064 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559f16b-231b-42f6-9776-7c529650eecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:58.771911 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559f16b-231b-42f6-9776-7c529650eecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:58.772621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559f16b-231b-42f6-9776-7c529650eecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:58.773039 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559f16b-231b-42f6-9776-7c529650eecf admin admin] PATH_INFO: `/v3/users/a9c89324e7d3449e9d805105e657ccab` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:58.774857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559f16b-231b-42f6-9776-7c529650eecf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9c89324e7d3449e9d805105e657ccab)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:58.779368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559f16b-231b-42f6-9776-7c529650eecf admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:58.837723 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:58.891368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b559f16b-231b-42f6-9776-7c529650eecf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9c89324e7d3449e9d805105e657ccab'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:58.901425 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b559f16b-231b-42f6-9776-7c529650eecf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9c89324e7d3449e9d805105e657ccab'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:58.903248 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b559f16b-231b-42f6-9776-7c529650eecf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9c89324e7d3449e9d805105e657ccab'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:58.924379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b559f16b-231b-42f6-9776-7c529650eecf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9c89324e7d3449e9d805105e657ccab'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:58.927438 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b559f16b-231b-42f6-9776-7c529650eecf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9c89324e7d3449e9d805105e657ccab'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:58.953933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1070/2123] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:39:58 2022] DELETE /identity/v3/users/a9c89324e7d3449e9d805105e657ccab => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:58.958349 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4680c-f310-4a01-b4c4-e32000bb8bba None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:58.958795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4680c-f310-4a01-b4c4-e32000bb8bba None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:58.959269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4680c-f310-4a01-b4c4-e32000bb8bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:58.970653 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:58.996529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-feb4680c-f310-4a01-b4c4-e32000bb8bba None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:58.997931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-feb4680c-f310-4a01-b4c4-e32000bb8bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:59.103145 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1054/2124] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:39:59.128330 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559f16b-231b-42f6-9776-7c529650eecf admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:59.130026 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b559f16b-231b-42f6-9776-7c529650eecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:59.199303 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4680c-f310-4a01-b4c4-e32000bb8bba None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:59.201808 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-feb4680c-f310-4a01-b4c4-e32000bb8bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:59.231819 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4680c-f310-4a01-b4c4-e32000bb8bba None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:59.244624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f0872d-4976-47fb-8791-5ccb8e11a2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:59.245303 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0872d-4976-47fb-8791-5ccb8e11a2fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:59.245925 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0872d-4976-47fb-8791-5ccb8e11a2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:59.246186 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0872d-4976-47fb-8791-5ccb8e11a2fa admin admin] PATH_INFO: `/v3/projects/0909643c71294b24b81d5cd767211e94` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:59.249352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f0872d-4976-47fb-8791-5ccb8e11a2fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=0909643c71294b24b81d5cd767211e94)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:59.253533 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f0872d-4976-47fb-8791-5ccb8e11a2fa admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:59.266825 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559f16b-231b-42f6-9776-7c529650eecf admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:59.280039 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9943127c-d211-41e5-819b-2fd381a7079a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:59.280747 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943127c-d211-41e5-819b-2fd381a7079a service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:59.281031 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943127c-d211-41e5-819b-2fd381a7079a service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:59.281716 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943127c-d211-41e5-819b-2fd381a7079a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:59.304680 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9943127c-d211-41e5-819b-2fd381a7079a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:59.307313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9943127c-d211-41e5-819b-2fd381a7079a service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:59.331892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1071/2125] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:39:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:39:59.358498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c1f0872d-4976-47fb-8791-5ccb8e11a2fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0909643c71294b24b81d5cd767211e94', 'request_id': 'req-c1f0872d-4976-47fb-8791-5ccb8e11a2fa'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:59.368332 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c1f0872d-4976-47fb-8791-5ccb8e11a2fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0909643c71294b24b81d5cd767211e94', 'request_id': 'req-c1f0872d-4976-47fb-8791-5ccb8e11a2fa'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:59.377072 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c1f0872d-4976-47fb-8791-5ccb8e11a2fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0909643c71294b24b81d5cd767211e94', 'request_id': 'req-c1f0872d-4976-47fb-8791-5ccb8e11a2fa'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:59.392827 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c1f0872d-4976-47fb-8791-5ccb8e11a2fa admin admin] The token cache is being invalidate because project 0909643c71294b24b81d5cd767211e94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:39:59.393106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c1f0872d-4976-47fb-8791-5ccb8e11a2fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:59.394244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c1f0872d-4976-47fb-8791-5ccb8e11a2fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:39:59.396091 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1055/2126] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:39:59 2022] DELETE /identity/v3/projects/0909643c71294b24b81d5cd767211e94 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:39:59.401973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856eb18-c7eb-4255-8a4a-7e71763c07b6 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:59.402478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856eb18-c7eb-4255-8a4a-7e71763c07b6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:59.402925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856eb18-c7eb-4255-8a4a-7e71763c07b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:59.429519 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-9856eb18-c7eb-4255-8a4a-7e71763c07b6 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:59.430376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9856eb18-c7eb-4255-8a4a-7e71763c07b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:59.518497 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1072/2127] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:59.528036 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89dc90-8047-4512-987d-bb4d75516e46 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:59.528570 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89dc90-8047-4512-987d-bb4d75516e46 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:59.528899 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89dc90-8047-4512-987d-bb4d75516e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:59.552946 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-2a89dc90-8047-4512-987d-bb4d75516e46 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:39:59.553822 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-2a89dc90-8047-4512-987d-bb4d75516e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:39:59.566547 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1056/2128] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:39:59.572218 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856eb18-c7eb-4255-8a4a-7e71763c07b6 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:59.582449 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856eb18-c7eb-4255-8a4a-7e71763c07b6 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:59.587615 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca1fff-bad3-4095-bcec-008096e3f7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:59.588139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca1fff-bad3-4095-bcec-008096e3f7fe admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:59.588418 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca1fff-bad3-4095-bcec-008096e3f7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:59.588691 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca1fff-bad3-4095-bcec-008096e3f7fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:59.589996 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca1fff-bad3-4095-bcec-008096e3f7fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:39:59.590956 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca1fff-bad3-4095-bcec-008096e3f7fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:59.592592 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca1fff-bad3-4095-bcec-008096e3f7fe admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:59.610889 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1073/2129] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:39:59.612278 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89dc90-8047-4512-987d-bb4d75516e46 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:59.622765 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89dc90-8047-4512-987d-bb4d75516e46 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:59.627981 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf5f0d-1a93-4329-927e-c2b132bddb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:59.628614 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf5f0d-1a93-4329-927e-c2b132bddb6f admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:59.628974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf5f0d-1a93-4329-927e-c2b132bddb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:59.629160 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf5f0d-1a93-4329-927e-c2b132bddb6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:59.629941 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf5f0d-1a93-4329-927e-c2b132bddb6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:59.632284 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf5f0d-1a93-4329-927e-c2b132bddb6f admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:59.665488 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1057/2130] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:39:59 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 04:39:59.674588 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca1fff-bad3-4095-bcec-008096e3f7fe admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:59.689606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca1fff-bad3-4095-bcec-008096e3f7fe admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:59.722808 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:39:59.778257 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e19fae-465f-45b0-8dd1-82c238e9c225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:59.779993 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e19fae-465f-45b0-8dd1-82c238e9c225 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:59.779993 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e19fae-465f-45b0-8dd1-82c238e9c225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:59.779993 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e19fae-465f-45b0-8dd1-82c238e9c225 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:59.781860 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e19fae-465f-45b0-8dd1-82c238e9c225 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:59.786472 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e19fae-465f-45b0-8dd1-82c238e9c225 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:59.831820 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1074/2131] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:39:59 2022] POST /identity/v3/users => generated 288 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:39:59.832576 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf5f0d-1a93-4329-927e-c2b132bddb6f admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:59.846422 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf5f0d-1a93-4329-927e-c2b132bddb6f admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:59.859605 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe1dd0-497d-4fe4-9b5f-929eb8a78086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:59.859861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe1dd0-497d-4fe4-9b5f-929eb8a78086 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:59.860136 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe1dd0-497d-4fe4-9b5f-929eb8a78086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:59.860359 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe1dd0-497d-4fe4-9b5f-929eb8a78086 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:59.861257 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe1dd0-497d-4fe4-9b5f-929eb8a78086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:59.862877 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe1dd0-497d-4fe4-9b5f-929eb8a78086 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:59.898970 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1058/2132] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 04:39:59.904478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e19fae-465f-45b0-8dd1-82c238e9c225 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:59.924936 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e19fae-465f-45b0-8dd1-82c238e9c225 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:59.934226 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4d423-f64d-4aba-9140-085409c6312d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:59.934893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4d423-f64d-4aba-9140-085409c6312d admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:59.935254 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4d423-f64d-4aba-9140-085409c6312d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:59.935603 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4d423-f64d-4aba-9140-085409c6312d admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:39:59.936526 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4d423-f64d-4aba-9140-085409c6312d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:39:59.938390 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4d423-f64d-4aba-9140-085409c6312d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:39:59.960614 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1075/2133] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:39:59.961499 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe1dd0-497d-4fe4-9b5f-929eb8a78086 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:39:59.975622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe1dd0-497d-4fe4-9b5f-929eb8a78086 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:39:59.982684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65398ec6-4635-4aa0-af20-baa131212dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:39:59.983389 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65398ec6-4635-4aa0-af20-baa131212dd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:39:59.983675 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65398ec6-4635-4aa0-af20-baa131212dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:39:59.983955 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65398ec6-4635-4aa0-af20-baa131212dd6 admin admin] PATH_INFO: `/v3/projects/0e350730438549bd963cf0ada2723aca/users/a9426fee66694033924f45edb3c7e744/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:00.005904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65398ec6-4635-4aa0-af20-baa131212dd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e350730438549bd963cf0ada2723aca, user_id=a9426fee66694033924f45edb3c7e744, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:00.009363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65398ec6-4635-4aa0-af20-baa131212dd6 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:00.028256 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1059/2134] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:39:59 2022] PUT /identity/v3/projects/0e350730438549bd963cf0ada2723aca/users/a9426fee66694033924f45edb3c7e744/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:40:00.045140 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2261d-c834-4d8a-b805-412ee7d4b0ce None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:00.045634 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2261d-c834-4d8a-b805-412ee7d4b0ce None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:00.046079 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2261d-c834-4d8a-b805-412ee7d4b0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:00.083779 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-9be2261d-c834-4d8a-b805-412ee7d4b0ce None None] MFA Rules not processed for user `a9426fee66694033924f45edb3c7e744`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:00.085285 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9be2261d-c834-4d8a-b805-412ee7d4b0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:00.170188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1076/2135] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:00.173795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65398ec6-4635-4aa0-af20-baa131212dd6 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:00.212687 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:00.213070 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2261d-c834-4d8a-b805-412ee7d4b0ce None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:00.216392 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9be2261d-c834-4d8a-b805-412ee7d4b0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:00.308074 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65398ec6-4635-4aa0-af20-baa131212dd6 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:00.314183 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11efe3f0-c318-4d60-98f6-9171c2d4f799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:00.315086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efe3f0-c318-4d60-98f6-9171c2d4f799 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:00.315476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efe3f0-c318-4d60-98f6-9171c2d4f799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:00.315970 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efe3f0-c318-4d60-98f6-9171c2d4f799 admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:00.317121 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11efe3f0-c318-4d60-98f6-9171c2d4f799 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:00.319436 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11efe3f0-c318-4d60-98f6-9171c2d4f799 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:00.331144 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2261d-c834-4d8a-b805-412ee7d4b0ce None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:00.342856 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95377142-fe19-481d-b05e-3756b10abe18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:00.343576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95377142-fe19-481d-b05e-3756b10abe18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:00.343774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95377142-fe19-481d-b05e-3756b10abe18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:00.344104 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95377142-fe19-481d-b05e-3756b10abe18 admin admin] PATH_INFO: `/v3/projects/e2032f1adcd94de18d309bce9becf7eb` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:00.345727 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95377142-fe19-481d-b05e-3756b10abe18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2032f1adcd94de18d309bce9becf7eb)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:00.348170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95377142-fe19-481d-b05e-3756b10abe18 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:00.369298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1060/2136] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:40:00 2022] POST /identity/v3/projects => generated 375 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:40:00.369298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7008dd9-bad5-4fe9-8148-e91f49c07bf9 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:00.369298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7008dd9-bad5-4fe9-8148-e91f49c07bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:00.369298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7008dd9-bad5-4fe9-8148-e91f49c07bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:00.439600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-e7008dd9-bad5-4fe9-8148-e91f49c07bf9 None None] MFA Rules not processed for user `1b23dd1311554910a19399aa74df9e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:00.441112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-e7008dd9-bad5-4fe9-8148-e91f49c07bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:00.452802 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:00.470921 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-95377142-fe19-481d-b05e-3756b10abe18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2032f1adcd94de18d309bce9becf7eb', 'request_id': 'req-95377142-fe19-481d-b05e-3756b10abe18'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:00.485044 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-95377142-fe19-481d-b05e-3756b10abe18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2032f1adcd94de18d309bce9becf7eb', 'request_id': 'req-95377142-fe19-481d-b05e-3756b10abe18'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:00.494092 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-95377142-fe19-481d-b05e-3756b10abe18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2032f1adcd94de18d309bce9becf7eb', 'request_id': 'req-95377142-fe19-481d-b05e-3756b10abe18'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:00.533914 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-95377142-fe19-481d-b05e-3756b10abe18 admin admin] The token cache is being invalidate because project e2032f1adcd94de18d309bce9becf7eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:40:00.534212 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-95377142-fe19-481d-b05e-3756b10abe18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:00.536159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-95377142-fe19-481d-b05e-3756b10abe18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:00.539131 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1077/2137] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:40:00 2022] DELETE /identity/v3/projects/e2032f1adcd94de18d309bce9becf7eb => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:40:00.539902 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95377142-fe19-481d-b05e-3756b10abe18 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:00.542481 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-95377142-fe19-481d-b05e-3756b10abe18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:00.588677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1061/2138] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:00.590708 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7008dd9-bad5-4fe9-8148-e91f49c07bf9 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:00.592367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-e7008dd9-bad5-4fe9-8148-e91f49c07bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:00.689274 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95377142-fe19-481d-b05e-3756b10abe18 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:00.695992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e066f0c-f896-42bb-83b0-32cc78ecb89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:00.696634 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e066f0c-f896-42bb-83b0-32cc78ecb89c admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:00.696911 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e066f0c-f896-42bb-83b0-32cc78ecb89c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:00.697366 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e066f0c-f896-42bb-83b0-32cc78ecb89c admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:00.698096 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e066f0c-f896-42bb-83b0-32cc78ecb89c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:00.700621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e066f0c-f896-42bb-83b0-32cc78ecb89c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:00.716391 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7008dd9-bad5-4fe9-8148-e91f49c07bf9 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:00.729578 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b289a-70bc-43d0-ad23-74dfa6b13b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:00.729578 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b289a-70bc-43d0-ad23-74dfa6b13b7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:00.729892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b289a-70bc-43d0-ad23-74dfa6b13b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:00.730080 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b289a-70bc-43d0-ad23-74dfa6b13b7f admin admin] PATH_INFO: `/v3/users/353cfbda09e34082bbed614067aea039` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:00.730711 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1078/2139] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:40:00 2022] POST /identity/v3/users => generated 294 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:40:00.731680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b289a-70bc-43d0-ad23-74dfa6b13b7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=353cfbda09e34082bbed614067aea039)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:00.731795 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e066f0c-f896-42bb-83b0-32cc78ecb89c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:00.733426 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8e066f0c-f896-42bb-83b0-32cc78ecb89c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:00.733859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b289a-70bc-43d0-ad23-74dfa6b13b7f admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:00.811468 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:00.816147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e066f0c-f896-42bb-83b0-32cc78ecb89c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:00.820220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-381b289a-70bc-43d0-ad23-74dfa6b13b7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '353cfbda09e34082bbed614067aea039'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:00.820884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-381b289a-70bc-43d0-ad23-74dfa6b13b7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '353cfbda09e34082bbed614067aea039'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:00.821614 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf2961-1e19-4055-8978-81996adbe67f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:00.822261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf2961-1e19-4055-8978-81996adbe67f service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:00.822597 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf2961-1e19-4055-8978-81996adbe67f service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:00.823205 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf2961-1e19-4055-8978-81996adbe67f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:00.825735 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9eaf2961-1e19-4055-8978-81996adbe67f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:00.842918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf2961-1e19-4055-8978-81996adbe67f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:00.844871 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf2961-1e19-4055-8978-81996adbe67f service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:00.847785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-381b289a-70bc-43d0-ad23-74dfa6b13b7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '353cfbda09e34082bbed614067aea039'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:00.858019 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1079/2140] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:40:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:40:00.858993 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf2961-1e19-4055-8978-81996adbe67f service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:00.860943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9eaf2961-1e19-4055-8978-81996adbe67f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:00.865301 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-381b289a-70bc-43d0-ad23-74dfa6b13b7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '353cfbda09e34082bbed614067aea039'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:00.866027 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-381b289a-70bc-43d0-ad23-74dfa6b13b7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '353cfbda09e34082bbed614067aea039'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:00.889218 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1062/2141] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:40:00 2022] DELETE /identity/v3/users/353cfbda09e34082bbed614067aea039 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:00.890336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf2961-1e19-4055-8978-81996adbe67f service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:00.974047 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:00.988136 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f35035-b9bb-423b-b567-23fd79012d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:00.990382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f35035-b9bb-423b-b567-23fd79012d13 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:00.991774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f35035-b9bb-423b-b567-23fd79012d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:00.992544 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f35035-b9bb-423b-b567-23fd79012d13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:00.994778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f35035-b9bb-423b-b567-23fd79012d13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:00.999001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f35035-b9bb-423b-b567-23fd79012d13 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:01.042101 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1080/2142] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:01.048025 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b289a-70bc-43d0-ad23-74dfa6b13b7f admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:01.066884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b289a-70bc-43d0-ad23-74dfa6b13b7f admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:01.075436 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13bd0a-02b5-4612-a5fb-6db0ca8dddd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:01.077659 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13bd0a-02b5-4612-a5fb-6db0ca8dddd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:01.078181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13bd0a-02b5-4612-a5fb-6db0ca8dddd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:01.078710 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13bd0a-02b5-4612-a5fb-6db0ca8dddd6 admin admin] PATH_INFO: `/v3/projects/abb5d463425140969f0ae57e469aa36c/users/a15f7edd4a454fd9a3e2060f00cd8c88/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:01.115760 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13bd0a-02b5-4612-a5fb-6db0ca8dddd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb5d463425140969f0ae57e469aa36c, user_id=a15f7edd4a454fd9a3e2060f00cd8c88, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:01.119397 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13bd0a-02b5-4612-a5fb-6db0ca8dddd6 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:01.150585 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1063/2143] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:40:01 2022] PUT /identity/v3/projects/abb5d463425140969f0ae57e469aa36c/users/a15f7edd4a454fd9a3e2060f00cd8c88/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:01.152572 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f35035-b9bb-423b-b567-23fd79012d13 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:01.164362 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f35035-b9bb-423b-b567-23fd79012d13 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:01.274459 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9639d-994e-40f5-a66b-a722ca122f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:01.275864 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9639d-994e-40f5-a66b-a722ca122f6a admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:01.276311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9639d-994e-40f5-a66b-a722ca122f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:01.276595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9639d-994e-40f5-a66b-a722ca122f6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:01.277993 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9639d-994e-40f5-a66b-a722ca122f6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:01.280144 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9639d-994e-40f5-a66b-a722ca122f6a admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:01.300573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1081/2144] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:01.302631 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13bd0a-02b5-4612-a5fb-6db0ca8dddd6 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:01.306597 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-1f13bd0a-02b5-4612-a5fb-6db0ca8dddd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:01.320670 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9639d-994e-40f5-a66b-a722ca122f6a admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:01.354022 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13bd0a-02b5-4612-a5fb-6db0ca8dddd6 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:01.369684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9639d-994e-40f5-a66b-a722ca122f6a admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:01.381289 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958100a-0be7-4c1e-911c-3bb51111eed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:01.381919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958100a-0be7-4c1e-911c-3bb51111eed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:01.382206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958100a-0be7-4c1e-911c-3bb51111eed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:01.382504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958100a-0be7-4c1e-911c-3bb51111eed3 admin admin] PATH_INFO: `/v3/projects/b3efbf2bb946439d89f790737784f630` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:01.384054 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c958100a-0be7-4c1e-911c-3bb51111eed3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3efbf2bb946439d89f790737784f630)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:01.388350 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c958100a-0be7-4c1e-911c-3bb51111eed3 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:01.391472 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5abe119-de26-4754-8a46-623fba4b8fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:01.393389 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abe119-de26-4754-8a46-623fba4b8fa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:01.394015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abe119-de26-4754-8a46-623fba4b8fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:01.394703 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abe119-de26-4754-8a46-623fba4b8fa3 admin admin] PATH_INFO: `/v3/projects/abb5d463425140969f0ae57e469aa36c/users/a15f7edd4a454fd9a3e2060f00cd8c88/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:01.397294 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5abe119-de26-4754-8a46-623fba4b8fa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb5d463425140969f0ae57e469aa36c, user_id=a15f7edd4a454fd9a3e2060f00cd8c88, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:01.400629 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5abe119-de26-4754-8a46-623fba4b8fa3 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:01.415555 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1082/2145] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:40:01 2022] PUT /identity/v3/projects/abb5d463425140969f0ae57e469aa36c/users/a15f7edd4a454fd9a3e2060f00cd8c88/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:01.426688 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e924d-71c0-46ef-8268-6a813749e8bf None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:01.427010 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e924d-71c0-46ef-8268-6a813749e8bf None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:01.427354 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e924d-71c0-46ef-8268-6a813749e8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:01.463465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c958100a-0be7-4c1e-911c-3bb51111eed3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3efbf2bb946439d89f790737784f630', 'request_id': 'req-c958100a-0be7-4c1e-911c-3bb51111eed3'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:01.469081 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-e71e924d-71c0-46ef-8268-6a813749e8bf None None] MFA Rules not processed for user `a15f7edd4a454fd9a3e2060f00cd8c88`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:01.471365 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e71e924d-71c0-46ef-8268-6a813749e8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:01.475387 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c958100a-0be7-4c1e-911c-3bb51111eed3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3efbf2bb946439d89f790737784f630', 'request_id': 'req-c958100a-0be7-4c1e-911c-3bb51111eed3'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:01.486545 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c958100a-0be7-4c1e-911c-3bb51111eed3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3efbf2bb946439d89f790737784f630', 'request_id': 'req-c958100a-0be7-4c1e-911c-3bb51111eed3'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:01.504646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c958100a-0be7-4c1e-911c-3bb51111eed3 admin admin] The token cache is being invalidate because project b3efbf2bb946439d89f790737784f630 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:40:01.504935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c958100a-0be7-4c1e-911c-3bb51111eed3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:01.505691 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c958100a-0be7-4c1e-911c-3bb51111eed3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:01.507816 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1064/2146] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:40:01 2022] DELETE /identity/v3/projects/b3efbf2bb946439d89f790737784f630 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:01.515989 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02eff0c-fe61-4c7b-ae43-9b59ea4d3b54 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:01.516334 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02eff0c-fe61-4c7b-ae43-9b59ea4d3b54 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:01.516552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02eff0c-fe61-4c7b-ae43-9b59ea4d3b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:01.541449 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-a02eff0c-fe61-4c7b-ae43-9b59ea4d3b54 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:01.543974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-a02eff0c-fe61-4c7b-ae43-9b59ea4d3b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:01.570514 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1083/2147] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:40:01.583747 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e924d-71c0-46ef-8268-6a813749e8bf None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:01.585222 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e71e924d-71c0-46ef-8268-6a813749e8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:01.635383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1065/2148] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:01.639129 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407cbca-e122-4180-b0e2-6c4fed9d17a3 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:01.639420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407cbca-e122-4180-b0e2-6c4fed9d17a3 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:01.639765 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407cbca-e122-4180-b0e2-6c4fed9d17a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:01.661953 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-7407cbca-e122-4180-b0e2-6c4fed9d17a3 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:01.663168 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-7407cbca-e122-4180-b0e2-6c4fed9d17a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:01.671723 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e924d-71c0-46ef-8268-6a813749e8bf None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:01.676309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1066/2149] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:40:01.678428 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7407cbca-e122-4180-b0e2-6c4fed9d17a3 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:01.680296 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6231fb79-b9e4-411c-be24-fb1fa6f06b7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:01.681486 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231fb79-b9e4-411c-be24-fb1fa6f06b7b service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:01.682125 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231fb79-b9e4-411c-be24-fb1fa6f06b7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:01.682411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231fb79-b9e4-411c-be24-fb1fa6f06b7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:01.686147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6231fb79-b9e4-411c-be24-fb1fa6f06b7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:01.689108 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7407cbca-e122-4180-b0e2-6c4fed9d17a3 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:01.695933 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371cdd4d-c356-4983-8c7c-5807f99adada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:01.696568 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cdd4d-c356-4983-8c7c-5807f99adada admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:01.696850 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cdd4d-c356-4983-8c7c-5807f99adada admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:01.697120 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cdd4d-c356-4983-8c7c-5807f99adada admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:01.697854 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371cdd4d-c356-4983-8c7c-5807f99adada admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:40:01.698157 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371cdd4d-c356-4983-8c7c-5807f99adada admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:01.700010 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371cdd4d-c356-4983-8c7c-5807f99adada admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:01.704014 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:01.713587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:01.737586 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1067/2150] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:40:01.738681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371cdd4d-c356-4983-8c7c-5807f99adada admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:01.752059 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371cdd4d-c356-4983-8c7c-5807f99adada admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:01.759726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bc6c6-f790-43e1-b93d-417a571f5c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:01.760806 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bc6c6-f790-43e1-b93d-417a571f5c62 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:01.761194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bc6c6-f790-43e1-b93d-417a571f5c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:01.761662 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bc6c6-f790-43e1-b93d-417a571f5c62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:01.761840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6231fb79-b9e4-411c-be24-fb1fa6f06b7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:01.762422 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825bc6c6-f790-43e1-b93d-417a571f5c62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:01.763224 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6231fb79-b9e4-411c-be24-fb1fa6f06b7b service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:01.764498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825bc6c6-f790-43e1-b93d-417a571f5c62 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:01.773529 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1084/2151] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:40:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:40:01.802538 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1068/2152] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:40:01 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:40:01.806905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6231fb79-b9e4-411c-be24-fb1fa6f06b7b service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:01.820641 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6231fb79-b9e4-411c-be24-fb1fa6f06b7b service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:01.916188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0f5d5-4cf4-4f7f-916c-d5dfa05b48eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:01.916188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0f5d5-4cf4-4f7f-916c-d5dfa05b48eb admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:01.916188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0f5d5-4cf4-4f7f-916c-d5dfa05b48eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:01.916188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0f5d5-4cf4-4f7f-916c-d5dfa05b48eb admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:01.916655 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0f5d5-4cf4-4f7f-916c-d5dfa05b48eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:01.916655 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0f5d5-4cf4-4f7f-916c-d5dfa05b48eb admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:01.957780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1085/2153] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:40:01 2022] POST /identity/v3/users => generated 293 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:40:01.958385 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bc6c6-f790-43e1-b93d-417a571f5c62 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:01.969735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bc6c6-f790-43e1-b93d-417a571f5c62 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:01.975739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc1cc6-1817-4959-a0f9-a9cecd7d3aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:01.976451 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc1cc6-1817-4959-a0f9-a9cecd7d3aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:01.976728 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc1cc6-1817-4959-a0f9-a9cecd7d3aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:01.977570 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc1cc6-1817-4959-a0f9-a9cecd7d3aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:01.978776 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fc1cc6-1817-4959-a0f9-a9cecd7d3aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:01.980973 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fc1cc6-1817-4959-a0f9-a9cecd7d3aa7 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:01.993843 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1069/2154] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:01.999284 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0f5d5-4cf4-4f7f-916c-d5dfa05b48eb admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:02.015146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0f5d5-4cf4-4f7f-916c-d5dfa05b48eb admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:02.021152 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a2614-b332-4db7-98b4-385e19b92bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:02.021837 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a2614-b332-4db7-98b4-385e19b92bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:02.022114 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a2614-b332-4db7-98b4-385e19b92bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:02.022382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a2614-b332-4db7-98b4-385e19b92bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:02.023211 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a2614-b332-4db7-98b4-385e19b92bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:02.024680 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a2614-b332-4db7-98b4-385e19b92bc1 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:02.046674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1086/2155] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:02.048298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc1cc6-1817-4959-a0f9-a9cecd7d3aa7 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:02.065190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc1cc6-1817-4959-a0f9-a9cecd7d3aa7 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:02.071496 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177350ee-f367-4511-bd7b-cc168a9b390b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:02.072254 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177350ee-f367-4511-bd7b-cc168a9b390b admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:02.072608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177350ee-f367-4511-bd7b-cc168a9b390b admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:02.072957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177350ee-f367-4511-bd7b-cc168a9b390b admin admin] PATH_INFO: `/v3/projects/3a4e66510dc844908f92c4b4e7cf6425/users/3106addbdba542be91ef533fa4c751f3/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:02.098994 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177350ee-f367-4511-bd7b-cc168a9b390b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a4e66510dc844908f92c4b4e7cf6425, user_id=3106addbdba542be91ef533fa4c751f3, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:02.102361 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177350ee-f367-4511-bd7b-cc168a9b390b admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:02.138739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1070/2156] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:40:02 2022] PUT /identity/v3/projects/3a4e66510dc844908f92c4b4e7cf6425/users/3106addbdba542be91ef533fa4c751f3/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:02.151068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c470b-3045-40a6-979e-471ba35ce6a8 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:02.152560 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c470b-3045-40a6-979e-471ba35ce6a8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:02.152914 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c470b-3045-40a6-979e-471ba35ce6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:02.209906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-1a9c470b-3045-40a6-979e-471ba35ce6a8 None None] MFA Rules not processed for user `3106addbdba542be91ef533fa4c751f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:02.211920 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1a9c470b-3045-40a6-979e-471ba35ce6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:02.305443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1087/2157] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:02.310181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177350ee-f367-4511-bd7b-cc168a9b390b admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:02.380027 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177350ee-f367-4511-bd7b-cc168a9b390b admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:02.385361 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17966de9-e9d0-4908-b45b-ec3f49c816ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:02.385625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17966de9-e9d0-4908-b45b-ec3f49c816ee admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:02.385914 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17966de9-e9d0-4908-b45b-ec3f49c816ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:02.386717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17966de9-e9d0-4908-b45b-ec3f49c816ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:02.387859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17966de9-e9d0-4908-b45b-ec3f49c816ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:02.389819 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17966de9-e9d0-4908-b45b-ec3f49c816ee admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:02.423182 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1071/2158] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:40:02 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:40:02.429873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c470b-3045-40a6-979e-471ba35ce6a8 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:02.444747 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4ae42-fbd1-4c56-ba8c-d64ff1660ce4 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:02.445184 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4ae42-fbd1-4c56-ba8c-d64ff1660ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:02.445475 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4ae42-fbd1-4c56-ba8c-d64ff1660ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:02.445826 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c470b-3045-40a6-979e-471ba35ce6a8 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:02.469353 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-8ef4ae42-fbd1-4c56-ba8c-d64ff1660ce4 None None] MFA Rules not processed for user `1b23dd1311554910a19399aa74df9e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:02.471103 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-8ef4ae42-fbd1-4c56-ba8c-d64ff1660ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:02.483907 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:02.533465 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b7036-50ee-442b-bbf2-04eb7b047ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:02.534227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b7036-50ee-442b-bbf2-04eb7b047ed6 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:02.534580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b7036-50ee-442b-bbf2-04eb7b047ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:02.534926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b7036-50ee-442b-bbf2-04eb7b047ed6 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:02.535691 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b7036-50ee-442b-bbf2-04eb7b047ed6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:02.537726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b7036-50ee-442b-bbf2-04eb7b047ed6 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:02.569100 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1072/2159] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:40:02.573615 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1088/2160] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:40:02 2022] POST /identity/v3/users => generated 300 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:40:02.573808 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4ae42-fbd1-4c56-ba8c-d64ff1660ce4 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:02.586486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4ae42-fbd1-4c56-ba8c-d64ff1660ce4 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:02.600000 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586a18b-b42c-44fa-a17c-eaa52cc8bd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:02.600730 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586a18b-b42c-44fa-a17c-eaa52cc8bd7e admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:02.601020 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586a18b-b42c-44fa-a17c-eaa52cc8bd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:02.601298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586a18b-b42c-44fa-a17c-eaa52cc8bd7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:02.602114 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f586a18b-b42c-44fa-a17c-eaa52cc8bd7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:02.604825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f586a18b-b42c-44fa-a17c-eaa52cc8bd7e admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:02.618352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1073/2161] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:02.622851 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b7036-50ee-442b-bbf2-04eb7b047ed6 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:02.672436 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b7036-50ee-442b-bbf2-04eb7b047ed6 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:02.688807 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f468129-0751-461c-bf09-6a71ffe33618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:02.690735 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f468129-0751-461c-bf09-6a71ffe33618 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:02.690735 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f468129-0751-461c-bf09-6a71ffe33618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:02.690735 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f468129-0751-461c-bf09-6a71ffe33618 admin admin] PATH_INFO: `/v3/projects/3a36366505224c98b35d46b9afefffbd/users/971ddb19ab5f49dbbded0540ba649141/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:02.708564 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f468129-0751-461c-bf09-6a71ffe33618 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a36366505224c98b35d46b9afefffbd, user_id=971ddb19ab5f49dbbded0540ba649141, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:02.711364 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f468129-0751-461c-bf09-6a71ffe33618 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:02.736778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1089/2162] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:40:02 2022] PUT /identity/v3/projects/3a36366505224c98b35d46b9afefffbd/users/971ddb19ab5f49dbbded0540ba649141/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:40:02.739031 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586a18b-b42c-44fa-a17c-eaa52cc8bd7e admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:02.750007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586a18b-b42c-44fa-a17c-eaa52cc8bd7e admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:02.765060 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:02.823375 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a92d5-7932-4ccf-b4ac-ed89a5790cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:02.824786 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a92d5-7932-4ccf-b4ac-ed89a5790cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:02.825311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a92d5-7932-4ccf-b4ac-ed89a5790cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:02.825763 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a92d5-7932-4ccf-b4ac-ed89a5790cb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:02.826621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a92d5-7932-4ccf-b4ac-ed89a5790cb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:02.828355 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a92d5-7932-4ccf-b4ac-ed89a5790cb4 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:02.843605 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1074/2163] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:02.849145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f468129-0751-461c-bf09-6a71ffe33618 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:02.862948 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f468129-0751-461c-bf09-6a71ffe33618 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:02.871137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b5d64b-c912-4a77-9bf3-e02a6cb7460d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:02.872533 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5d64b-c912-4a77-9bf3-e02a6cb7460d admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:02.873785 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5d64b-c912-4a77-9bf3-e02a6cb7460d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:02.874557 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5d64b-c912-4a77-9bf3-e02a6cb7460d admin admin] PATH_INFO: `/v3/projects/3a36366505224c98b35d46b9afefffbd/users/971ddb19ab5f49dbbded0540ba649141/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:02.877315 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b5d64b-c912-4a77-9bf3-e02a6cb7460d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a36366505224c98b35d46b9afefffbd, user_id=971ddb19ab5f49dbbded0540ba649141, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:02.881143 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b5d64b-c912-4a77-9bf3-e02a6cb7460d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:02.908303 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1090/2164] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:40:02 2022] PUT /identity/v3/projects/3a36366505224c98b35d46b9afefffbd/users/971ddb19ab5f49dbbded0540ba649141/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:02.909300 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5884757-ea44-465d-8d4b-05afb34cc85c None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:02.909629 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5884757-ea44-465d-8d4b-05afb34cc85c None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:02.909910 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5884757-ea44-465d-8d4b-05afb34cc85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:02.943756 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-b5884757-ea44-465d-8d4b-05afb34cc85c None None] MFA Rules not processed for user `971ddb19ab5f49dbbded0540ba649141`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:02.945443 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-b5884757-ea44-465d-8d4b-05afb34cc85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:03.065970 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1075/2165] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:03.078159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b5d64b-c912-4a77-9bf3-e02a6cb7460d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:03.081184 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f6b5d64b-c912-4a77-9bf3-e02a6cb7460d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:03.157718 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b5d64b-c912-4a77-9bf3-e02a6cb7460d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:03.166398 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee189b-e51e-4418-9254-b4d6797a48a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:03.167806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee189b-e51e-4418-9254-b4d6797a48a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:03.168371 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee189b-e51e-4418-9254-b4d6797a48a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:03.169291 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee189b-e51e-4418-9254-b4d6797a48a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:03.182714 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee189b-e51e-4418-9254-b4d6797a48a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:03.185091 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee189b-e51e-4418-9254-b4d6797a48a4 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:03.221240 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1091/2166] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:40:03 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:40:04.023352 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5884757-ea44-465d-8d4b-05afb34cc85c None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:04.119817 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5884757-ea44-465d-8d4b-05afb34cc85c None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:04.126000 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59de695-1304-41b1-8a9f-b9006dc27b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:04.127130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59de695-1304-41b1-8a9f-b9006dc27b51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:04.127692 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59de695-1304-41b1-8a9f-b9006dc27b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:04.128202 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59de695-1304-41b1-8a9f-b9006dc27b51 admin admin] PATH_INFO: `/v3/users/3106addbdba542be91ef533fa4c751f3` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:04.129795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59de695-1304-41b1-8a9f-b9006dc27b51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3106addbdba542be91ef533fa4c751f3)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:04.131845 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59de695-1304-41b1-8a9f-b9006dc27b51 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:04.202874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:04.217149 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a59de695-1304-41b1-8a9f-b9006dc27b51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3106addbdba542be91ef533fa4c751f3'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:04.217923 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a59de695-1304-41b1-8a9f-b9006dc27b51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3106addbdba542be91ef533fa4c751f3'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:04.238315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a59de695-1304-41b1-8a9f-b9006dc27b51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3106addbdba542be91ef533fa4c751f3'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:04.251543 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a59de695-1304-41b1-8a9f-b9006dc27b51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3106addbdba542be91ef533fa4c751f3'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:04.252243 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a59de695-1304-41b1-8a9f-b9006dc27b51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3106addbdba542be91ef533fa4c751f3'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:04.269221 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1076/2167] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:40:04 2022] DELETE /identity/v3/users/3106addbdba542be91ef533fa4c751f3 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:04.275647 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e092651-9529-49d5-81ec-c6dbe4fd3461 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:04.276422 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e092651-9529-49d5-81ec-c6dbe4fd3461 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:04.276422 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e092651-9529-49d5-81ec-c6dbe4fd3461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:04.311776 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-1e092651-9529-49d5-81ec-c6dbe4fd3461 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:04.314680 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1e092651-9529-49d5-81ec-c6dbe4fd3461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:04.406902 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1092/2168] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:40:04.416808 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59de695-1304-41b1-8a9f-b9006dc27b51 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:04.420267 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-a59de695-1304-41b1-8a9f-b9006dc27b51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:04.494584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59de695-1304-41b1-8a9f-b9006dc27b51 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:04.500119 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368548f7-d92d-421d-adbf-5966a9850b9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:04.500846 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368548f7-d92d-421d-adbf-5966a9850b9e service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:04.501159 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368548f7-d92d-421d-adbf-5966a9850b9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:04.501401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368548f7-d92d-421d-adbf-5966a9850b9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:04.513606 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368548f7-d92d-421d-adbf-5966a9850b9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:04.515055 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368548f7-d92d-421d-adbf-5966a9850b9e service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:04.528157 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1077/2169] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:40:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:40:05.059573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e092651-9529-49d5-81ec-c6dbe4fd3461 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:05.061019 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1e092651-9529-49d5-81ec-c6dbe4fd3461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:05.074987 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e092651-9529-49d5-81ec-c6dbe4fd3461 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:05.080947 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3e3ad-a780-469d-9672-08e2af5ac1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:05.081718 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3e3ad-a780-469d-9672-08e2af5ac1c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:05.081999 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3e3ad-a780-469d-9672-08e2af5ac1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:05.082273 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3e3ad-a780-469d-9672-08e2af5ac1c7 admin admin] PATH_INFO: `/v3/projects/3a4e66510dc844908f92c4b4e7cf6425` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:05.083664 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3e3ad-a780-469d-9672-08e2af5ac1c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a4e66510dc844908f92c4b4e7cf6425)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:05.085801 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3e3ad-a780-469d-9672-08e2af5ac1c7 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:05.105197 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:05.157523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b3e3e3ad-a780-469d-9672-08e2af5ac1c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a4e66510dc844908f92c4b4e7cf6425', 'request_id': 'req-b3e3e3ad-a780-469d-9672-08e2af5ac1c7'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:05.170113 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b3e3e3ad-a780-469d-9672-08e2af5ac1c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a4e66510dc844908f92c4b4e7cf6425', 'request_id': 'req-b3e3e3ad-a780-469d-9672-08e2af5ac1c7'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:05.180316 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b3e3e3ad-a780-469d-9672-08e2af5ac1c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a4e66510dc844908f92c4b4e7cf6425', 'request_id': 'req-b3e3e3ad-a780-469d-9672-08e2af5ac1c7'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:05.196256 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b3e3e3ad-a780-469d-9672-08e2af5ac1c7 admin admin] The token cache is being invalidate because project 3a4e66510dc844908f92c4b4e7cf6425 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:40:05.196517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b3e3e3ad-a780-469d-9672-08e2af5ac1c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:05.197332 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b3e3e3ad-a780-469d-9672-08e2af5ac1c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:05.200566 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1093/2170] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:40:05 2022] DELETE /identity/v3/projects/3a4e66510dc844908f92c4b4e7cf6425 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:05.204016 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368548f7-d92d-421d-adbf-5966a9850b9e service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:05.215676 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368548f7-d92d-421d-adbf-5966a9850b9e service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:05.295724 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8748a17-a5db-4dc8-8c59-cca4d2a19dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:05.296498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8748a17-a5db-4dc8-8c59-cca4d2a19dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:05.296908 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8748a17-a5db-4dc8-8c59-cca4d2a19dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:05.297311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8748a17-a5db-4dc8-8c59-cca4d2a19dec admin admin] PATH_INFO: `/v3/users/971ddb19ab5f49dbbded0540ba649141` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:05.298776 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8748a17-a5db-4dc8-8c59-cca4d2a19dec admin admin] RBAC: Authorizing `identity:delete_user(user_id=971ddb19ab5f49dbbded0540ba649141)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:05.300801 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8748a17-a5db-4dc8-8c59-cca4d2a19dec admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:05.355683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3e3ad-a780-469d-9672-08e2af5ac1c7 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:05.357944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b3e3e3ad-a780-469d-9672-08e2af5ac1c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:05.373778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3e3ad-a780-469d-9672-08e2af5ac1c7 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:05.374394 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d8748a17-a5db-4dc8-8c59-cca4d2a19dec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '971ddb19ab5f49dbbded0540ba649141'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:05.375177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d8748a17-a5db-4dc8-8c59-cca4d2a19dec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '971ddb19ab5f49dbbded0540ba649141'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:05.379281 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f81ae-d637-4ee5-b158-a20fb2a02650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:05.380092 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f81ae-d637-4ee5-b158-a20fb2a02650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:05.380261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f81ae-d637-4ee5-b158-a20fb2a02650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:05.380500 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f81ae-d637-4ee5-b158-a20fb2a02650 admin admin] PATH_INFO: `/v3/users/99d00c09d8374f59ac1260ade95be9ec` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:05.381751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f81ae-d637-4ee5-b158-a20fb2a02650 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99d00c09d8374f59ac1260ade95be9ec)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:05.383674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f81ae-d637-4ee5-b158-a20fb2a02650 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:05.388464 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:05.390266 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d8748a17-a5db-4dc8-8c59-cca4d2a19dec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '971ddb19ab5f49dbbded0540ba649141'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:05.404320 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d8748a17-a5db-4dc8-8c59-cca4d2a19dec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '971ddb19ab5f49dbbded0540ba649141'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:05.404962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d8748a17-a5db-4dc8-8c59-cca4d2a19dec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '971ddb19ab5f49dbbded0540ba649141'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:05.420818 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1078/2171] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:40:05 2022] DELETE /identity/v3/users/971ddb19ab5f49dbbded0540ba649141 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:05.435110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2f5f81ae-d637-4ee5-b158-a20fb2a02650 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99d00c09d8374f59ac1260ade95be9ec'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:05.459990 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2f5f81ae-d637-4ee5-b158-a20fb2a02650 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99d00c09d8374f59ac1260ade95be9ec'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:05.461031 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2f5f81ae-d637-4ee5-b158-a20fb2a02650 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99d00c09d8374f59ac1260ade95be9ec'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:05.498511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2f5f81ae-d637-4ee5-b158-a20fb2a02650 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99d00c09d8374f59ac1260ade95be9ec'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:05.499514 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2f5f81ae-d637-4ee5-b158-a20fb2a02650 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99d00c09d8374f59ac1260ade95be9ec'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:05.521382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1094/2172] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:40:05 2022] DELETE /identity/v3/users/99d00c09d8374f59ac1260ade95be9ec => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:05.984855 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8748a17-a5db-4dc8-8c59-cca4d2a19dec admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:05.986402 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d8748a17-a5db-4dc8-8c59-cca4d2a19dec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:06.078446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8748a17-a5db-4dc8-8c59-cca4d2a19dec admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:06.183419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcc719-a09c-4e7d-9cf5-ebfa7c1a1bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:06.184362 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcc719-a09c-4e7d-9cf5-ebfa7c1a1bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:06.184638 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcc719-a09c-4e7d-9cf5-ebfa7c1a1bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:06.184941 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcc719-a09c-4e7d-9cf5-ebfa7c1a1bc6 admin admin] PATH_INFO: `/v3/projects/a89819385e8b42bb89e40449dd21ca86` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:06.186285 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bcc719-a09c-4e7d-9cf5-ebfa7c1a1bc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a89819385e8b42bb89e40449dd21ca86)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:06.188163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bcc719-a09c-4e7d-9cf5-ebfa7c1a1bc6 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:06.195230 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f81ae-d637-4ee5-b158-a20fb2a02650 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:06.197997 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2f5f81ae-d637-4ee5-b158-a20fb2a02650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:06.232210 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f81ae-d637-4ee5-b158-a20fb2a02650 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:06.240777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e7efe-116c-4260-8692-3adfe44aad60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:06.254128 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e7efe-116c-4260-8692-3adfe44aad60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:06.254580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e7efe-116c-4260-8692-3adfe44aad60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:06.254895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e7efe-116c-4260-8692-3adfe44aad60 admin admin] PATH_INFO: `/v3/projects/3a36366505224c98b35d46b9afefffbd` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:06.257936 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e7efe-116c-4260-8692-3adfe44aad60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a36366505224c98b35d46b9afefffbd)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:06.258298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-36bcc719-a09c-4e7d-9cf5-ebfa7c1a1bc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a89819385e8b42bb89e40449dd21ca86', 'request_id': 'req-36bcc719-a09c-4e7d-9cf5-ebfa7c1a1bc6'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:06.259580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e7efe-116c-4260-8692-3adfe44aad60 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:06.271084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-36bcc719-a09c-4e7d-9cf5-ebfa7c1a1bc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a89819385e8b42bb89e40449dd21ca86', 'request_id': 'req-36bcc719-a09c-4e7d-9cf5-ebfa7c1a1bc6'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:06.280795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-36bcc719-a09c-4e7d-9cf5-ebfa7c1a1bc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a89819385e8b42bb89e40449dd21ca86', 'request_id': 'req-36bcc719-a09c-4e7d-9cf5-ebfa7c1a1bc6'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:06.298701 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-36bcc719-a09c-4e7d-9cf5-ebfa7c1a1bc6 admin admin] The token cache is being invalidate because project a89819385e8b42bb89e40449dd21ca86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:40:06.299183 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-36bcc719-a09c-4e7d-9cf5-ebfa7c1a1bc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:06.300005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-36bcc719-a09c-4e7d-9cf5-ebfa7c1a1bc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:06.305324 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1079/2173] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:40:05 2022] DELETE /identity/v3/projects/a89819385e8b42bb89e40449dd21ca86 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:06.308571 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a018c5-e692-45ea-a0b9-96fd24620a24 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:06.309095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a018c5-e692-45ea-a0b9-96fd24620a24 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:06.309456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a018c5-e692-45ea-a0b9-96fd24620a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:06.331225 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-cc4e7efe-116c-4260-8692-3adfe44aad60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a36366505224c98b35d46b9afefffbd', 'request_id': 'req-cc4e7efe-116c-4260-8692-3adfe44aad60'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:06.336842 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-93a018c5-e692-45ea-a0b9-96fd24620a24 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:06.338182 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-93a018c5-e692-45ea-a0b9-96fd24620a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:06.345241 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-cc4e7efe-116c-4260-8692-3adfe44aad60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a36366505224c98b35d46b9afefffbd', 'request_id': 'req-cc4e7efe-116c-4260-8692-3adfe44aad60'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:06.352378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-cc4e7efe-116c-4260-8692-3adfe44aad60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a36366505224c98b35d46b9afefffbd', 'request_id': 'req-cc4e7efe-116c-4260-8692-3adfe44aad60'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:06.360083 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:06.365690 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=73,keystone:DELETE=24,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:06.379196 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-cc4e7efe-116c-4260-8692-3adfe44aad60 admin admin] The token cache is being invalidate because project 3a36366505224c98b35d46b9afefffbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:40:06.379949 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-cc4e7efe-116c-4260-8692-3adfe44aad60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:06.381187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-cc4e7efe-116c-4260-8692-3adfe44aad60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:06.389686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1095/2174] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:40:06 2022] DELETE /identity/v3/projects/3a36366505224c98b35d46b9afefffbd => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:06.394581 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038523e-36c8-4d45-a6ad-55eaefb3484a None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:06.394943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038523e-36c8-4d45-a6ad-55eaefb3484a None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:06.395313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038523e-36c8-4d45-a6ad-55eaefb3484a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:06.420556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-2038523e-36c8-4d45-a6ad-55eaefb3484a None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:06.421815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2038523e-36c8-4d45-a6ad-55eaefb3484a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:06.475254 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1080/2175] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:06.486955 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c1918-b590-46cd-b605-005d0ddcfdfc None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:06.487507 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c1918-b590-46cd-b605-005d0ddcfdfc None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:06.487937 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c1918-b590-46cd-b605-005d0ddcfdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:06.513687 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-7f5c1918-b590-46cd-b605-005d0ddcfdfc None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:06.514932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-7f5c1918-b590-46cd-b605-005d0ddcfdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:06.520892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1096/2176] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:06.529426 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1081/2177] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:40:06.529840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda9aba-2361-428b-b9b6-bb31a14057bc None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:06.530134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda9aba-2361-428b-b9b6-bb31a14057bc None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:06.530697 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda9aba-2361-428b-b9b6-bb31a14057bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:06.533593 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c1918-b590-46cd-b605-005d0ddcfdfc None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:06.544029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c1918-b590-46cd-b605-005d0ddcfdfc None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:06.550898 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14df78-8754-4c17-9164-ee1abc5cbd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:06.551704 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14df78-8754-4c17-9164-ee1abc5cbd69 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:06.552481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14df78-8754-4c17-9164-ee1abc5cbd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:06.552858 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14df78-8754-4c17-9164-ee1abc5cbd69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:06.553714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f14df78-8754-4c17-9164-ee1abc5cbd69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:40:06.554124 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f14df78-8754-4c17-9164-ee1abc5cbd69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:06.554326 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-9eda9aba-2361-428b-b9b6-bb31a14057bc None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:06.555561 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9eda9aba-2361-428b-b9b6-bb31a14057bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:06.556934 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f14df78-8754-4c17-9164-ee1abc5cbd69 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:06.570189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1097/2178] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:40:06.576132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda9aba-2361-428b-b9b6-bb31a14057bc None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:06.577457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1082/2179] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:40:06.581706 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14df78-8754-4c17-9164-ee1abc5cbd69 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:06.586531 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda9aba-2361-428b-b9b6-bb31a14057bc None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:06.594902 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14df78-8754-4c17-9164-ee1abc5cbd69 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:06.596391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eedc1e9-858d-4bca-945c-0172f3c42c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:06.597632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedc1e9-858d-4bca-945c-0172f3c42c9a admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:06.597946 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedc1e9-858d-4bca-945c-0172f3c42c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:06.598336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedc1e9-858d-4bca-945c-0172f3c42c9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:06.599501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eedc1e9-858d-4bca-945c-0172f3c42c9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:40:06.599805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eedc1e9-858d-4bca-945c-0172f3c42c9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:06.600617 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c1272-6ee6-4e53-801b-3430aab3e7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:06.601286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c1272-6ee6-4e53-801b-3430aab3e7f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:06.601715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eedc1e9-858d-4bca-945c-0172f3c42c9a admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:06.601844 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c1272-6ee6-4e53-801b-3430aab3e7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:06.602017 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c1272-6ee6-4e53-801b-3430aab3e7f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:06.602834 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c1272-6ee6-4e53-801b-3430aab3e7f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:06.607307 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c1272-6ee6-4e53-801b-3430aab3e7f6 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:06.619504 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1098/2180] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:40:06.619932 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eedc1e9-858d-4bca-945c-0172f3c42c9a admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:06.630841 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eedc1e9-858d-4bca-945c-0172f3c42c9a admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:06.638871 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1083/2181] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:40:06 2022] POST /identity/v3/projects => generated 393 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 04:40:06.639038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108e34d-0122-4305-85c9-6beca67d989f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:06.639038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108e34d-0122-4305-85c9-6beca67d989f admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:06.639038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108e34d-0122-4305-85c9-6beca67d989f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:06.639038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108e34d-0122-4305-85c9-6beca67d989f admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:06.639338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108e34d-0122-4305-85c9-6beca67d989f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:06.639526 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108e34d-0122-4305-85c9-6beca67d989f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:06.639862 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c1272-6ee6-4e53-801b-3430aab3e7f6 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:06.676455 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1099/2182] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:40:06 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:40:06.678278 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108e34d-0122-4305-85c9-6beca67d989f admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:06.719001 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c1272-6ee6-4e53-801b-3430aab3e7f6 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:06.756672 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108e34d-0122-4305-85c9-6beca67d989f admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:06.762349 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f11c9-c277-4db7-8a91-c277acbe77d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:06.762981 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f11c9-c277-4db7-8a91-c277acbe77d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:06.763255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f11c9-c277-4db7-8a91-c277acbe77d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:06.763527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f11c9-c277-4db7-8a91-c277acbe77d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:06.764343 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f11c9-c277-4db7-8a91-c277acbe77d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:06.766308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f11c9-c277-4db7-8a91-c277acbe77d2 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:06.788139 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0717d1-fd0b-467c-8e75-054569f8772c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:06.788771 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0717d1-fd0b-467c-8e75-054569f8772c admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:06.789052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0717d1-fd0b-467c-8e75-054569f8772c admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:06.789325 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0717d1-fd0b-467c-8e75-054569f8772c admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:06.789849 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1100/2183] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:40:06 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:40:06.790033 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0717d1-fd0b-467c-8e75-054569f8772c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:06.790693 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f11c9-c277-4db7-8a91-c277acbe77d2 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:06.791956 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0717d1-fd0b-467c-8e75-054569f8772c admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:06.793194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-982f11c9-c277-4db7-8a91-c277acbe77d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:06.816467 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1084/2184] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:40:06 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 04:40:06.821051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0717d1-fd0b-467c-8e75-054569f8772c admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:06.834243 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0717d1-fd0b-467c-8e75-054569f8772c admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:06.840505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c3748-9a7b-4329-a771-09266f66c580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:06.841135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c3748-9a7b-4329-a771-09266f66c580 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:06.841418 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c3748-9a7b-4329-a771-09266f66c580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:06.841718 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c3748-9a7b-4329-a771-09266f66c580 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:06.842427 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144c3748-9a7b-4329-a771-09266f66c580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:06.844030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144c3748-9a7b-4329-a771-09266f66c580 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:06.861180 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1085/2185] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:06.861180 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c3748-9a7b-4329-a771-09266f66c580 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:06.865960 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:06.873074 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f11c9-c277-4db7-8a91-c277acbe77d2 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:06.876118 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:06.879173 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31859e5-7905-40ff-8982-5d1c93439c43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:06.879812 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31859e5-7905-40ff-8982-5d1c93439c43 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:06.880148 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31859e5-7905-40ff-8982-5d1c93439c43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:06.880561 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31859e5-7905-40ff-8982-5d1c93439c43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:06.882951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c31859e5-7905-40ff-8982-5d1c93439c43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:06.884393 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c3748-9a7b-4329-a771-09266f66c580 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:06.894946 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ce31b-2d86-4b08-b800-3015581ecebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:06.895587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ce31b-2d86-4b08-b800-3015581ecebd admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:06.895875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ce31b-2d86-4b08-b800-3015581ecebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:06.896151 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ce31b-2d86-4b08-b800-3015581ecebd admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:06.896863 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51ce31b-2d86-4b08-b800-3015581ecebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:06.898537 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51ce31b-2d86-4b08-b800-3015581ecebd admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:06.913025 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1086/2186] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:06.913025 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ce31b-2d86-4b08-b800-3015581ecebd admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:06.924497 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ce31b-2d86-4b08-b800-3015581ecebd admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:06.929640 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfa5152-59cd-4475-a48d-af3f51e19bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:06.929915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa5152-59cd-4475-a48d-af3f51e19bcc admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:06.929915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa5152-59cd-4475-a48d-af3f51e19bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:06.929915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa5152-59cd-4475-a48d-af3f51e19bcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:06.931170 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfa5152-59cd-4475-a48d-af3f51e19bcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:06.933353 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfa5152-59cd-4475-a48d-af3f51e19bcc admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:06.946965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1087/2187] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:06.946965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfa5152-59cd-4475-a48d-af3f51e19bcc admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:06.955580 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfa5152-59cd-4475-a48d-af3f51e19bcc admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:06.960316 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe051e8e-5ca5-4332-b86a-ba94c2c6b89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:06.960990 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe051e8e-5ca5-4332-b86a-ba94c2c6b89e admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:06.961270 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe051e8e-5ca5-4332-b86a-ba94c2c6b89e admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:06.961602 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe051e8e-5ca5-4332-b86a-ba94c2c6b89e admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:06.962311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe051e8e-5ca5-4332-b86a-ba94c2c6b89e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:06.963930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe051e8e-5ca5-4332-b86a-ba94c2c6b89e admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:06.968693 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31859e5-7905-40ff-8982-5d1c93439c43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:06.970189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31859e5-7905-40ff-8982-5d1c93439c43 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:06.974412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1088/2188] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:06.975593 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe051e8e-5ca5-4332-b86a-ba94c2c6b89e admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:06.982403 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1101/2189] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:40:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:40:06.983161 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31859e5-7905-40ff-8982-5d1c93439c43 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:06.986643 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe051e8e-5ca5-4332-b86a-ba94c2c6b89e admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:06.992317 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13440e-673b-4cbf-a4be-9dab5e74972d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:06.992994 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13440e-673b-4cbf-a4be-9dab5e74972d admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:06.993274 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13440e-673b-4cbf-a4be-9dab5e74972d admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:06.993568 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13440e-673b-4cbf-a4be-9dab5e74972d admin admin] PATH_INFO: `/v3/projects/04e96b6523d9418997d83e1e57a4425e/users/35f3d17087dc4748b241350e76034ce8/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:06.994125 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31859e5-7905-40ff-8982-5d1c93439c43 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:07.005267 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa4301-fd4e-488b-9139-63cc132331d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:07.006063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa4301-fd4e-488b-9139-63cc132331d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:07.006348 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa4301-fd4e-488b-9139-63cc132331d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:07.006624 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa4301-fd4e-488b-9139-63cc132331d6 admin admin] PATH_INFO: `/v3/projects/e43171bfd4564b0c92395767641b91ce/users/40a2cbbc654749199e742ae96329a1d9/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:07.019066 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13440e-673b-4cbf-a4be-9dab5e74972d admin admin] RBAC: Authorizing `identity:create_grant(project_id=04e96b6523d9418997d83e1e57a4425e, user_id=35f3d17087dc4748b241350e76034ce8, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:07.023374 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13440e-673b-4cbf-a4be-9dab5e74972d admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:07.031164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa4301-fd4e-488b-9139-63cc132331d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e43171bfd4564b0c92395767641b91ce, user_id=40a2cbbc654749199e742ae96329a1d9, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:07.034341 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa4301-fd4e-488b-9139-63cc132331d6 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:07.046436 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1089/2190] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:40:06 2022] PUT /identity/v3/projects/04e96b6523d9418997d83e1e57a4425e/users/35f3d17087dc4748b241350e76034ce8/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:07.052251 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97eb14-6f85-4bf7-afea-3d8879dfdad6 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:07.055990 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1102/2191] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:40:06 2022] PUT /identity/v3/projects/e43171bfd4564b0c92395767641b91ce/users/40a2cbbc654749199e742ae96329a1d9/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:07.056165 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97eb14-6f85-4bf7-afea-3d8879dfdad6 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:07.056165 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97eb14-6f85-4bf7-afea-3d8879dfdad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:07.059788 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e27cc-bb4d-498f-aade-e715a92dc03a None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:07.060137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e27cc-bb4d-498f-aade-e715a92dc03a None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:07.060577 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e27cc-bb4d-498f-aade-e715a92dc03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:07.077571 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-dd97eb14-6f85-4bf7-afea-3d8879dfdad6 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:07.079873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-dd97eb14-6f85-4bf7-afea-3d8879dfdad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:07.088518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-0f8e27cc-bb4d-498f-aade-e715a92dc03a None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:07.090882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0f8e27cc-bb4d-498f-aade-e715a92dc03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:07.234323 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1090/2192] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:07.243621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1103/2193] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:07.255855 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97eb14-6f85-4bf7-afea-3d8879dfdad6 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:07.257168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e27cc-bb4d-498f-aade-e715a92dc03a None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:07.257534 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-dd97eb14-6f85-4bf7-afea-3d8879dfdad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:07.262544 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0f8e27cc-bb4d-498f-aade-e715a92dc03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:07.356997 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e27cc-bb4d-498f-aade-e715a92dc03a None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:07.365635 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b7650-b6a8-4876-ac88-d914b494e983 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:07.366274 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b7650-b6a8-4876-ac88-d914b494e983 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:07.366854 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b7650-b6a8-4876-ac88-d914b494e983 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:07.366854 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b7650-b6a8-4876-ac88-d914b494e983 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:07.385852 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b7650-b6a8-4876-ac88-d914b494e983 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:07.387093 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b7650-b6a8-4876-ac88-d914b494e983 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:07.392586 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97eb14-6f85-4bf7-afea-3d8879dfdad6 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:07.397830 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1577a4-5bb5-43bd-b2cb-9dab9145b747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:07.398563 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1577a4-5bb5-43bd-b2cb-9dab9145b747 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:07.398917 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1577a4-5bb5-43bd-b2cb-9dab9145b747 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:07.399266 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1577a4-5bb5-43bd-b2cb-9dab9145b747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:07.409012 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1104/2194] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:40:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:40:07.415749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1577a4-5bb5-43bd-b2cb-9dab9145b747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:07.419348 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1577a4-5bb5-43bd-b2cb-9dab9145b747 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:07.435495 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1091/2195] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:40:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:40:08.791351 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d74fed-38f8-44db-a84c-52e775adf116 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:08.792286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d74fed-38f8-44db-a84c-52e775adf116 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:08.792567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d74fed-38f8-44db-a84c-52e775adf116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:08.839607 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-58d74fed-38f8-44db-a84c-52e775adf116 None None] MFA Rules not processed for user `35f3d17087dc4748b241350e76034ce8`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:08.843486 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-58d74fed-38f8-44db-a84c-52e775adf116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:08.851046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:08.925865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1105/2196] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:08.929901 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1577a4-5bb5-43bd-b2cb-9dab9145b747 service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:08.942815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1577a4-5bb5-43bd-b2cb-9dab9145b747 service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:08.953280 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20915351-cdb6-418d-a245-c1fef2726c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:08.954065 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20915351-cdb6-418d-a245-c1fef2726c4c admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:08.954359 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20915351-cdb6-418d-a245-c1fef2726c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:08.954637 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20915351-cdb6-418d-a245-c1fef2726c4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:08.955373 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20915351-cdb6-418d-a245-c1fef2726c4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:08.957434 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20915351-cdb6-418d-a245-c1fef2726c4c admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:08.997932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1092/2197] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:40:08 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:40:09.001958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d74fed-38f8-44db-a84c-52e775adf116 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:09.014627 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d74fed-38f8-44db-a84c-52e775adf116 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:09.115285 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ae70e-84a8-4ad4-bdef-9843b1ece404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:09.116124 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ae70e-84a8-4ad4-bdef-9843b1ece404 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:09.116837 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ae70e-84a8-4ad4-bdef-9843b1ece404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:09.117404 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ae70e-84a8-4ad4-bdef-9843b1ece404 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:09.118728 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ae70e-84a8-4ad4-bdef-9843b1ece404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:09.123111 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ae70e-84a8-4ad4-bdef-9843b1ece404 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:09.155307 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1106/2198] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:40:09 2022] POST /identity/v3/users => generated 293 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:40:09.157291 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20915351-cdb6-418d-a245-c1fef2726c4c admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:09.176543 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20915351-cdb6-418d-a245-c1fef2726c4c admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:09.186672 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3857c-7e00-4048-8069-b32ba3b8a908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:09.187485 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3857c-7e00-4048-8069-b32ba3b8a908 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:09.187774 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3857c-7e00-4048-8069-b32ba3b8a908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:09.188188 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3857c-7e00-4048-8069-b32ba3b8a908 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:09.189536 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d3857c-7e00-4048-8069-b32ba3b8a908 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:09.191056 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d3857c-7e00-4048-8069-b32ba3b8a908 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:09.209074 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1093/2199] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:09.210413 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ae70e-84a8-4ad4-bdef-9843b1ece404 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:09.226237 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ae70e-84a8-4ad4-bdef-9843b1ece404 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:09.235233 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742915c-bc47-4da5-8717-0d7e9086479f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:09.237023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742915c-bc47-4da5-8717-0d7e9086479f admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:09.237571 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742915c-bc47-4da5-8717-0d7e9086479f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:09.238913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742915c-bc47-4da5-8717-0d7e9086479f admin admin] PATH_INFO: `/v3/projects/6308adc367df42b9805d3556a5a95b6b/users/342740bcb7c44209978dfd38dc8de785/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:09.264315 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742915c-bc47-4da5-8717-0d7e9086479f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6308adc367df42b9805d3556a5a95b6b, user_id=342740bcb7c44209978dfd38dc8de785, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:09.267133 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742915c-bc47-4da5-8717-0d7e9086479f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:09.287954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1107/2200] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:40:09 2022] PUT /identity/v3/projects/6308adc367df42b9805d3556a5a95b6b/users/342740bcb7c44209978dfd38dc8de785/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:40:09.293354 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3857c-7e00-4048-8069-b32ba3b8a908 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:09.303892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3857c-7e00-4048-8069-b32ba3b8a908 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:09.323094 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:09.380158 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cafcb3-913c-4f2e-857c-787e71641449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:09.380872 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cafcb3-913c-4f2e-857c-787e71641449 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:09.381228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cafcb3-913c-4f2e-857c-787e71641449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:09.381872 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cafcb3-913c-4f2e-857c-787e71641449 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:09.384202 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cafcb3-913c-4f2e-857c-787e71641449 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:09.384202 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cafcb3-913c-4f2e-857c-787e71641449 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:09.394954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1094/2201] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:09.399138 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742915c-bc47-4da5-8717-0d7e9086479f admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:09.411254 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742915c-bc47-4da5-8717-0d7e9086479f admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:09.421609 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01820ad6-c731-4e3a-965d-d04c4c588aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:09.421609 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01820ad6-c731-4e3a-965d-d04c4c588aba admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:09.421609 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01820ad6-c731-4e3a-965d-d04c4c588aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:09.421609 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01820ad6-c731-4e3a-965d-d04c4c588aba admin admin] PATH_INFO: `/v3/projects/6308adc367df42b9805d3556a5a95b6b/users/342740bcb7c44209978dfd38dc8de785/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:09.422918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01820ad6-c731-4e3a-965d-d04c4c588aba admin admin] RBAC: Authorizing `identity:create_grant(project_id=6308adc367df42b9805d3556a5a95b6b, user_id=342740bcb7c44209978dfd38dc8de785, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:09.426361 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01820ad6-c731-4e3a-965d-d04c4c588aba admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:09.447983 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1108/2202] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:40:09 2022] PUT /identity/v3/projects/6308adc367df42b9805d3556a5a95b6b/users/342740bcb7c44209978dfd38dc8de785/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:11.347713 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a6e80-4d07-4d50-bc8e-83bfb057d2ed None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:11.348286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a6e80-4d07-4d50-bc8e-83bfb057d2ed None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:11.348429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a6e80-4d07-4d50-bc8e-83bfb057d2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:11.416382 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-d92a6e80-4d07-4d50-bc8e-83bfb057d2ed None None] MFA Rules not processed for user `342740bcb7c44209978dfd38dc8de785`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:11.419519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d92a6e80-4d07-4d50-bc8e-83bfb057d2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:11.567406 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1095/2203] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:11.580852 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01820ad6-c731-4e3a-965d-d04c4c588aba admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:11.582987 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-01820ad6-c731-4e3a-965d-d04c4c588aba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:11.679297 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01820ad6-c731-4e3a-965d-d04c4c588aba admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:11.684317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e36fb-2e4b-45a2-a521-3735595c4137 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:11.685684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e36fb-2e4b-45a2-a521-3735595c4137 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:11.686138 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e36fb-2e4b-45a2-a521-3735595c4137 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:11.686554 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e36fb-2e4b-45a2-a521-3735595c4137 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:11.712572 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:11.809405 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e36fb-2e4b-45a2-a521-3735595c4137 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:11.810995 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e36fb-2e4b-45a2-a521-3735595c4137 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:11.825002 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1109/2204] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:40:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:40:13.453143 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a6e80-4d07-4d50-bc8e-83bfb057d2ed None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:13.477019 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a6e80-4d07-4d50-bc8e-83bfb057d2ed None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:13.482368 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4925cc1a-c75c-4b5d-9159-ae468ffe2ac3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:13.483337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925cc1a-c75c-4b5d-9159-ae468ffe2ac3 service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:13.483795 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925cc1a-c75c-4b5d-9159-ae468ffe2ac3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:13.484269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925cc1a-c75c-4b5d-9159-ae468ffe2ac3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:13.509569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4925cc1a-c75c-4b5d-9159-ae468ffe2ac3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:13.512510 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4925cc1a-c75c-4b5d-9159-ae468ffe2ac3 service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:13.525828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1096/2205] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:40:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:40:14.036342 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527327c6-7321-4300-9bc4-308318465172 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:14.036846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527327c6-7321-4300-9bc4-308318465172 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:14.036946 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527327c6-7321-4300-9bc4-308318465172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:14.069286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-527327c6-7321-4300-9bc4-308318465172 None None] MFA Rules not processed for user `40a2cbbc654749199e742ae96329a1d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:14.070554 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-527327c6-7321-4300-9bc4-308318465172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:14.162925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1110/2206] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:14.174716 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45905e52-5adb-4980-8f97-91ff0e9fe94a None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:14.175118 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45905e52-5adb-4980-8f97-91ff0e9fe94a None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:14.175469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45905e52-5adb-4980-8f97-91ff0e9fe94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:14.196787 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-45905e52-5adb-4980-8f97-91ff0e9fe94a None None] MFA Rules not processed for user `40a2cbbc654749199e742ae96329a1d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:14.198659 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-45905e52-5adb-4980-8f97-91ff0e9fe94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:14.206736 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1097/2207] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:14.219893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527327c6-7321-4300-9bc4-308318465172 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:14.221392 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-527327c6-7321-4300-9bc4-308318465172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:14.311158 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527327c6-7321-4300-9bc4-308318465172 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:14.316902 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a00b8-9555-4047-a721-9588cc0a0a4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:14.317581 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a00b8-9555-4047-a721-9588cc0a0a4a service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:14.317864 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a00b8-9555-4047-a721-9588cc0a0a4a service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:14.318147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a00b8-9555-4047-a721-9588cc0a0a4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:14.328241 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a00b8-9555-4047-a721-9588cc0a0a4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:14.330097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a00b8-9555-4047-a721-9588cc0a0a4a service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:14.341588 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1111/2208] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:40:14 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:40:15.273650 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d28bd9-5c39-4bb7-b1c6-e603ba047ced None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:15.274606 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d28bd9-5c39-4bb7-b1c6-e603ba047ced None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:15.275080 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d28bd9-5c39-4bb7-b1c6-e603ba047ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:15.303612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-38d28bd9-5c39-4bb7-b1c6-e603ba047ced None None] MFA Rules not processed for user `40a2cbbc654749199e742ae96329a1d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:15.305940 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-38d28bd9-5c39-4bb7-b1c6-e603ba047ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:15.316374 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1098/2209] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:19.617085 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4589d1-f573-4fe1-8555-bda08c1e1747 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:19.617632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4589d1-f573-4fe1-8555-bda08c1e1747 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:19.618302 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4589d1-f573-4fe1-8555-bda08c1e1747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:19.661377 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ae4589d1-f573-4fe1-8555-bda08c1e1747 None None] MFA Rules not processed for user `3a721708c7e443e799349add27de8f84`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:19.663806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ae4589d1-f573-4fe1-8555-bda08c1e1747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:19.697355 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:19.747781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1112/2210] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Aug 4 04:40:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:19.800791 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d28bd9-5c39-4bb7-b1c6-e603ba047ced None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:19.803155 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-38d28bd9-5c39-4bb7-b1c6-e603ba047ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:19.895528 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d28bd9-5c39-4bb7-b1c6-e603ba047ced None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:19.903240 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195adf9d-cbe4-46ec-98aa-805b6d14533e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1e539006166434790823b839bd745d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:19.905861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195adf9d-cbe4-46ec-98aa-805b6d14533e service swift] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:19.905861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195adf9d-cbe4-46ec-98aa-805b6d14533e service swift] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:19.905861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195adf9d-cbe4-46ec-98aa-805b6d14533e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:19.915431 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195adf9d-cbe4-46ec-98aa-805b6d14533e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:19.917765 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195adf9d-cbe4-46ec-98aa-805b6d14533e service swift] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:19.923088 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:19.929735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1099/2211] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Aug 4 04:40:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 04:40:20.687370 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7940509-aec3-4e57-bfa9-fb119d83ded6 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:20.688104 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7940509-aec3-4e57-bfa9-fb119d83ded6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:20.688400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7940509-aec3-4e57-bfa9-fb119d83ded6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:20.709849 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b7940509-aec3-4e57-bfa9-fb119d83ded6 None None] MFA Rules not processed for user `3a721708c7e443e799349add27de8f84`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:20.711033 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b7940509-aec3-4e57-bfa9-fb119d83ded6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:20.722690 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1113/2212] 10.222.0.47 () {58 vars in 967 bytes} [Thu Aug 4 04:40:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:20.737107 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195adf9d-cbe4-46ec-98aa-805b6d14533e service swift] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:20.749669 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195adf9d-cbe4-46ec-98aa-805b6d14533e service swift] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:20.755540 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01335394-ac93-4612-aa1f-764a0a28610b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1e539006166434790823b839bd745d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:20.756172 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01335394-ac93-4612-aa1f-764a0a28610b service swift] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:20.756455 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01335394-ac93-4612-aa1f-764a0a28610b service swift] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:20.756734 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01335394-ac93-4612-aa1f-764a0a28610b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:20.766767 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01335394-ac93-4612-aa1f-764a0a28610b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:20.768381 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01335394-ac93-4612-aa1f-764a0a28610b service swift] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:20.777092 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1100/2213] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Aug 4 04:40:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 04:40:21.044649 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb24e7-8140-4220-a838-2ac7d4a07026 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:21.044934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb24e7-8140-4220-a838-2ac7d4a07026 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:21.045209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb24e7-8140-4220-a838-2ac7d4a07026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:21.065485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-4dfb24e7-8140-4220-a838-2ac7d4a07026 None None] MFA Rules not processed for user `3a721708c7e443e799349add27de8f84`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:21.066752 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4dfb24e7-8140-4220-a838-2ac7d4a07026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:21.074562 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1114/2214] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Aug 4 04:40:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:21.087002 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01335394-ac93-4612-aa1f-764a0a28610b service swift] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:21.098227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01335394-ac93-4612-aa1f-764a0a28610b service swift] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:21.103637 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3a0ad-1285-4a0d-b626-32b21e656f85 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1e539006166434790823b839bd745d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:21.104796 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3a0ad-1285-4a0d-b626-32b21e656f85 service swift] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:21.105129 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3a0ad-1285-4a0d-b626-32b21e656f85 service swift] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:21.105350 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3a0ad-1285-4a0d-b626-32b21e656f85 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:21.114463 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f3a0ad-1285-4a0d-b626-32b21e656f85 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:21.116039 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f3a0ad-1285-4a0d-b626-32b21e656f85 service swift] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:21.125714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1101/2215] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Aug 4 04:40:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 04:40:21.478709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147a434-0a0b-4536-91fe-331359cfbcfa None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:21.479966 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147a434-0a0b-4536-91fe-331359cfbcfa None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:21.480450 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147a434-0a0b-4536-91fe-331359cfbcfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:21.504427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-c147a434-0a0b-4536-91fe-331359cfbcfa None None] MFA Rules not processed for user `3a721708c7e443e799349add27de8f84`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:21.504427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c147a434-0a0b-4536-91fe-331359cfbcfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:21.508435 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1115/2216] 10.222.0.47 () {58 vars in 967 bytes} [Thu Aug 4 04:40:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:21.521122 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3a0ad-1285-4a0d-b626-32b21e656f85 service swift] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:21.532529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3a0ad-1285-4a0d-b626-32b21e656f85 service swift] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:21.537631 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b6bfb-63c4-4711-a7c3-a79b6d517c5f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1e539006166434790823b839bd745d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:21.538235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b6bfb-63c4-4711-a7c3-a79b6d517c5f service swift] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:21.538515 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b6bfb-63c4-4711-a7c3-a79b6d517c5f service swift] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:21.538810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b6bfb-63c4-4711-a7c3-a79b6d517c5f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:21.548233 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b6bfb-63c4-4711-a7c3-a79b6d517c5f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:21.549831 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b6bfb-63c4-4711-a7c3-a79b6d517c5f service swift] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:21.558762 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1102/2217] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Aug 4 04:40:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 04:40:23.737788 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c147a434-0a0b-4536-91fe-331359cfbcfa None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:23.739416 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c147a434-0a0b-4536-91fe-331359cfbcfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:23.812781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c147a434-0a0b-4536-91fe-331359cfbcfa None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:23.817522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57866e7e-89e3-4eea-8ced-e5089bdc7fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:23.818225 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57866e7e-89e3-4eea-8ced-e5089bdc7fc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:23.818671 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57866e7e-89e3-4eea-8ced-e5089bdc7fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:23.819571 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57866e7e-89e3-4eea-8ced-e5089bdc7fc0 admin admin] PATH_INFO: `/v3/users/13679676132642b88b2c88b58c4afc55` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:23.820873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57866e7e-89e3-4eea-8ced-e5089bdc7fc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13679676132642b88b2c88b58c4afc55)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:23.823431 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57866e7e-89e3-4eea-8ced-e5089bdc7fc0 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:23.875721 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-57866e7e-89e3-4eea-8ced-e5089bdc7fc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13679676132642b88b2c88b58c4afc55'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:23.883556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-57866e7e-89e3-4eea-8ced-e5089bdc7fc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13679676132642b88b2c88b58c4afc55'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:23.884281 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-57866e7e-89e3-4eea-8ced-e5089bdc7fc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13679676132642b88b2c88b58c4afc55'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:23.907510 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-57866e7e-89e3-4eea-8ced-e5089bdc7fc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13679676132642b88b2c88b58c4afc55'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:23.908329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-57866e7e-89e3-4eea-8ced-e5089bdc7fc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13679676132642b88b2c88b58c4afc55'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:23.931498 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1116/2218] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:40:23 2022] DELETE /identity/v3/users/13679676132642b88b2c88b58c4afc55 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:23.936989 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b21b93-5263-4fd5-8307-c598ae6b9b72 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:23.937798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b21b93-5263-4fd5-8307-c598ae6b9b72 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:23.937798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b21b93-5263-4fd5-8307-c598ae6b9b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:23.959493 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-04b21b93-5263-4fd5-8307-c598ae6b9b72 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:23.961494 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-04b21b93-5263-4fd5-8307-c598ae6b9b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:24.046872 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1103/2219] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:24.056973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57866e7e-89e3-4eea-8ced-e5089bdc7fc0 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:24.058939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-57866e7e-89e3-4eea-8ced-e5089bdc7fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:24.108966 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:24.162809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57866e7e-89e3-4eea-8ced-e5089bdc7fc0 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:24.168123 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de239dd0-ed39-4ac5-a99b-312abace9cff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:24.169891 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de239dd0-ed39-4ac5-a99b-312abace9cff service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:24.170187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de239dd0-ed39-4ac5-a99b-312abace9cff service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:24.170472 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de239dd0-ed39-4ac5-a99b-312abace9cff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:24.186175 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de239dd0-ed39-4ac5-a99b-312abace9cff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:24.188117 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de239dd0-ed39-4ac5-a99b-312abace9cff service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:24.201633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1117/2220] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:40:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:40:24.609113 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b21b93-5263-4fd5-8307-c598ae6b9b72 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:24.610699 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-04b21b93-5263-4fd5-8307-c598ae6b9b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:24.624741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b21b93-5263-4fd5-8307-c598ae6b9b72 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:24.630128 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c8f87-9400-4520-9320-66b004b46b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:24.630988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c8f87-9400-4520-9320-66b004b46b5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:24.631333 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c8f87-9400-4520-9320-66b004b46b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:24.631822 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c8f87-9400-4520-9320-66b004b46b5d admin admin] PATH_INFO: `/v3/projects/4f04dacfa01e42539204572c99bed3f6` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:24.635573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c8f87-9400-4520-9320-66b004b46b5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f04dacfa01e42539204572c99bed3f6)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:24.636923 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c8f87-9400-4520-9320-66b004b46b5d admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:24.718547 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d70c8f87-9400-4520-9320-66b004b46b5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f04dacfa01e42539204572c99bed3f6', 'request_id': 'req-d70c8f87-9400-4520-9320-66b004b46b5d'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:24.728730 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d70c8f87-9400-4520-9320-66b004b46b5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f04dacfa01e42539204572c99bed3f6', 'request_id': 'req-d70c8f87-9400-4520-9320-66b004b46b5d'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:24.734968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d70c8f87-9400-4520-9320-66b004b46b5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f04dacfa01e42539204572c99bed3f6', 'request_id': 'req-d70c8f87-9400-4520-9320-66b004b46b5d'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:24.746584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d70c8f87-9400-4520-9320-66b004b46b5d admin admin] The token cache is being invalidate because project 4f04dacfa01e42539204572c99bed3f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:40:24.746868 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d70c8f87-9400-4520-9320-66b004b46b5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:24.747515 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d70c8f87-9400-4520-9320-66b004b46b5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:24.749013 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1104/2221] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:40:24 2022] DELETE /identity/v3/projects/4f04dacfa01e42539204572c99bed3f6 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:24.753944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4042cad-1b9d-4426-9319-8fd03d9ca5e9 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:24.754392 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4042cad-1b9d-4426-9319-8fd03d9ca5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:24.754497 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4042cad-1b9d-4426-9319-8fd03d9ca5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:24.773982 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-f4042cad-1b9d-4426-9319-8fd03d9ca5e9 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:24.775156 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f4042cad-1b9d-4426-9319-8fd03d9ca5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:24.833313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1118/2222] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:40:24.841385 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420884a-9899-4748-9dd6-8f87a61a41ea None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:24.841698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420884a-9899-4748-9dd6-8f87a61a41ea None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:24.841974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420884a-9899-4748-9dd6-8f87a61a41ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:24.861087 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-a420884a-9899-4748-9dd6-8f87a61a41ea None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:24.862305 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-a420884a-9899-4748-9dd6-8f87a61a41ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:24.871808 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1105/2223] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:24.876154 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4042cad-1b9d-4426-9319-8fd03d9ca5e9 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:24.886325 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4042cad-1b9d-4426-9319-8fd03d9ca5e9 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:24.890859 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35750e-522e-4619-9ef5-319f175ca116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:24.891484 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35750e-522e-4619-9ef5-319f175ca116 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:24.891760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35750e-522e-4619-9ef5-319f175ca116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:24.892029 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35750e-522e-4619-9ef5-319f175ca116 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:24.892724 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35750e-522e-4619-9ef5-319f175ca116 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:40:24.893023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35750e-522e-4619-9ef5-319f175ca116 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:24.894655 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35750e-522e-4619-9ef5-319f175ca116 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:24.906928 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1119/2224] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:40:24.911255 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420884a-9899-4748-9dd6-8f87a61a41ea None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:24.920943 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420884a-9899-4748-9dd6-8f87a61a41ea None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:24.925812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d7c92f-f601-47f6-9a8f-86a5e0425484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:24.926419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7c92f-f601-47f6-9a8f-86a5e0425484 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:24.926694 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7c92f-f601-47f6-9a8f-86a5e0425484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:24.926961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7c92f-f601-47f6-9a8f-86a5e0425484 admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:24.927629 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d7c92f-f601-47f6-9a8f-86a5e0425484 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:24.929599 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d7c92f-f601-47f6-9a8f-86a5e0425484 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:24.957042 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1106/2225] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:40:24 2022] POST /identity/v3/projects => generated 397 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 04:40:24.961242 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35750e-522e-4619-9ef5-319f175ca116 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:24.972201 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35750e-522e-4619-9ef5-319f175ca116 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:25.032606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf32a5d-cea5-410f-88b2-aabf14ed8486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:25.033222 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf32a5d-cea5-410f-88b2-aabf14ed8486 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:25.033512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf32a5d-cea5-410f-88b2-aabf14ed8486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:25.033820 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf32a5d-cea5-410f-88b2-aabf14ed8486 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:25.034490 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf32a5d-cea5-410f-88b2-aabf14ed8486 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:25.036427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf32a5d-cea5-410f-88b2-aabf14ed8486 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:25.059759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1120/2226] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:40:24 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:40:25.065753 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d7c92f-f601-47f6-9a8f-86a5e0425484 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:25.078820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d7c92f-f601-47f6-9a8f-86a5e0425484 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:25.083556 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae4725-4c04-4787-93c6-dc0bc96383c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:25.084172 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae4725-4c04-4787-93c6-dc0bc96383c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:25.084451 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae4725-4c04-4787-93c6-dc0bc96383c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:25.084719 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae4725-4c04-4787-93c6-dc0bc96383c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:25.085414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae4725-4c04-4787-93c6-dc0bc96383c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:25.087020 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae4725-4c04-4787-93c6-dc0bc96383c9 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:25.099429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1107/2227] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:25.103529 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf32a5d-cea5-410f-88b2-aabf14ed8486 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:25.113784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf32a5d-cea5-410f-88b2-aabf14ed8486 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:25.118385 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e57964-77e2-4012-a306-288f771556db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:25.118992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e57964-77e2-4012-a306-288f771556db admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:25.119854 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e57964-77e2-4012-a306-288f771556db admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:25.120130 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e57964-77e2-4012-a306-288f771556db admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:25.120831 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e57964-77e2-4012-a306-288f771556db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:25.122438 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e57964-77e2-4012-a306-288f771556db admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:25.136277 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1121/2228] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:25.140595 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae4725-4c04-4787-93c6-dc0bc96383c9 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:25.147404 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:25.152468 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae4725-4c04-4787-93c6-dc0bc96383c9 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:25.160348 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a549af9-0d50-4548-86e2-8db2cf598f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:25.161009 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a549af9-0d50-4548-86e2-8db2cf598f56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:25.161508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a549af9-0d50-4548-86e2-8db2cf598f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:25.161882 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a549af9-0d50-4548-86e2-8db2cf598f56 admin admin] PATH_INFO: `/v3/projects/46680cd77cea47f381ea6e71e0d26fcb/users/6d2b603f28e24598ab14e11993984d6f/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:25.178521 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a549af9-0d50-4548-86e2-8db2cf598f56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46680cd77cea47f381ea6e71e0d26fcb, user_id=6d2b603f28e24598ab14e11993984d6f, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:25.181427 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a549af9-0d50-4548-86e2-8db2cf598f56 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:25.196388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1108/2229] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:40:25 2022] PUT /identity/v3/projects/46680cd77cea47f381ea6e71e0d26fcb/users/6d2b603f28e24598ab14e11993984d6f/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:40:25.208482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d6c9f-c42c-43d9-b557-0028a22693f6 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:25.208772 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d6c9f-c42c-43d9-b557-0028a22693f6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:25.209045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d6c9f-c42c-43d9-b557-0028a22693f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:25.239378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-786d6c9f-c42c-43d9-b557-0028a22693f6 None None] MFA Rules not processed for user `6d2b603f28e24598ab14e11993984d6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:25.240776 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-786d6c9f-c42c-43d9-b557-0028a22693f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:25.271728 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:25.309817 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1122/2230] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:25 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:25.321199 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a549af9-0d50-4548-86e2-8db2cf598f56 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:25.323768 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5a549af9-0d50-4548-86e2-8db2cf598f56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:25.391259 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a549af9-0d50-4548-86e2-8db2cf598f56 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:25.394996 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4db5d-f209-46a2-bae8-52c4830f5f7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:25.395623 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4db5d-f209-46a2-bae8-52c4830f5f7e service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:25.395908 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4db5d-f209-46a2-bae8-52c4830f5f7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:25.396184 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4db5d-f209-46a2-bae8-52c4830f5f7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:25.406009 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4db5d-f209-46a2-bae8-52c4830f5f7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:25.407443 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4db5d-f209-46a2-bae8-52c4830f5f7e service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:25.417196 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1109/2231] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:40:25 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:40:33.474094 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d6c9f-c42c-43d9-b557-0028a22693f6 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:33.475633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-786d6c9f-c42c-43d9-b557-0028a22693f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:33.588345 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d6c9f-c42c-43d9-b557-0028a22693f6 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:33.609214 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321f0e5-9e08-4ddc-a46f-d6f179504987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:33.609921 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321f0e5-9e08-4ddc-a46f-d6f179504987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:33.610204 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321f0e5-9e08-4ddc-a46f-d6f179504987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:33.610481 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321f0e5-9e08-4ddc-a46f-d6f179504987 admin admin] PATH_INFO: `/v3/users/a9426fee66694033924f45edb3c7e744` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:33.612507 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321f0e5-9e08-4ddc-a46f-d6f179504987 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9426fee66694033924f45edb3c7e744)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:33.615241 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321f0e5-9e08-4ddc-a46f-d6f179504987 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:33.747489 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0321f0e5-9e08-4ddc-a46f-d6f179504987 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9426fee66694033924f45edb3c7e744'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:33.783310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0321f0e5-9e08-4ddc-a46f-d6f179504987 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9426fee66694033924f45edb3c7e744'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:33.787284 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0321f0e5-9e08-4ddc-a46f-d6f179504987 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9426fee66694033924f45edb3c7e744'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:33.847787 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0321f0e5-9e08-4ddc-a46f-d6f179504987 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9426fee66694033924f45edb3c7e744'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:33.849712 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0321f0e5-9e08-4ddc-a46f-d6f179504987 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9426fee66694033924f45edb3c7e744'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:33.893912 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1123/2232] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:40:33 2022] DELETE /identity/v3/users/a9426fee66694033924f45edb3c7e744 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:33.902548 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769ede9-46cb-41c0-b9e5-b40c939179ae None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:33.903739 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769ede9-46cb-41c0-b9e5-b40c939179ae None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:33.904418 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769ede9-46cb-41c0-b9e5-b40c939179ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:33.952208 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-6769ede9-46cb-41c0-b9e5-b40c939179ae None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:33.953786 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-6769ede9-46cb-41c0-b9e5-b40c939179ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:34.068901 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1110/2233] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:34.081616 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321f0e5-9e08-4ddc-a46f-d6f179504987 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:34.083244 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0321f0e5-9e08-4ddc-a46f-d6f179504987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:34.168576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:34.176627 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321f0e5-9e08-4ddc-a46f-d6f179504987 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:34.184541 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d622d-6dbc-4c3a-abd2-c8736c957895 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:34.185198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d622d-6dbc-4c3a-abd2-c8736c957895 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:34.186346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d622d-6dbc-4c3a-abd2-c8736c957895 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:34.186953 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d622d-6dbc-4c3a-abd2-c8736c957895 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:34.199897 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d622d-6dbc-4c3a-abd2-c8736c957895 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:34.201289 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d622d-6dbc-4c3a-abd2-c8736c957895 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:34.211881 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1124/2234] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:40:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:40:34.723517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769ede9-46cb-41c0-b9e5-b40c939179ae None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:34.726912 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-6769ede9-46cb-41c0-b9e5-b40c939179ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:34.744572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769ede9-46cb-41c0-b9e5-b40c939179ae None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:34.755777 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb2a6c-4d04-4a7c-9196-c4908e7ad3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:34.757021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb2a6c-4d04-4a7c-9196-c4908e7ad3a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:34.757539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb2a6c-4d04-4a7c-9196-c4908e7ad3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:34.758314 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb2a6c-4d04-4a7c-9196-c4908e7ad3a0 admin admin] PATH_INFO: `/v3/projects/0e350730438549bd963cf0ada2723aca` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:34.759648 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb2a6c-4d04-4a7c-9196-c4908e7ad3a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e350730438549bd963cf0ada2723aca)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:34.763541 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb2a6c-4d04-4a7c-9196-c4908e7ad3a0 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:34.820563 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-69cb2a6c-4d04-4a7c-9196-c4908e7ad3a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e350730438549bd963cf0ada2723aca', 'request_id': 'req-69cb2a6c-4d04-4a7c-9196-c4908e7ad3a0'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:34.832174 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-69cb2a6c-4d04-4a7c-9196-c4908e7ad3a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e350730438549bd963cf0ada2723aca', 'request_id': 'req-69cb2a6c-4d04-4a7c-9196-c4908e7ad3a0'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:34.839575 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-69cb2a6c-4d04-4a7c-9196-c4908e7ad3a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e350730438549bd963cf0ada2723aca', 'request_id': 'req-69cb2a6c-4d04-4a7c-9196-c4908e7ad3a0'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:34.851107 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-69cb2a6c-4d04-4a7c-9196-c4908e7ad3a0 admin admin] The token cache is being invalidate because project 0e350730438549bd963cf0ada2723aca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:40:34.851393 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-69cb2a6c-4d04-4a7c-9196-c4908e7ad3a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:34.852069 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-69cb2a6c-4d04-4a7c-9196-c4908e7ad3a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:34.856087 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1111/2235] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:40:34 2022] DELETE /identity/v3/projects/0e350730438549bd963cf0ada2723aca => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:34.858650 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d622d-6dbc-4c3a-abd2-c8736c957895 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:34.869991 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d622d-6dbc-4c3a-abd2-c8736c957895 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:34.887332 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb2a6c-4d04-4a7c-9196-c4908e7ad3a0 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:34.888957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-69cb2a6c-4d04-4a7c-9196-c4908e7ad3a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:34.900416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:34.976200 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5a006-f2de-4477-baa4-8fc1ff6ecd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:34.976200 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5a006-f2de-4477-baa4-8fc1ff6ecd3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:34.976515 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5a006-f2de-4477-baa4-8fc1ff6ecd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:34.976740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5a006-f2de-4477-baa4-8fc1ff6ecd3a admin admin] PATH_INFO: `/v3/users/a15f7edd4a454fd9a3e2060f00cd8c88` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:34.978674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5a006-f2de-4477-baa4-8fc1ff6ecd3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a15f7edd4a454fd9a3e2060f00cd8c88)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:34.980758 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5a006-f2de-4477-baa4-8fc1ff6ecd3a admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:35.000774 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb2a6c-4d04-4a7c-9196-c4908e7ad3a0 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:35.006071 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830476c9-c5fd-4cab-a764-3fb1293eaee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:35.007120 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830476c9-c5fd-4cab-a764-3fb1293eaee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:35.007508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830476c9-c5fd-4cab-a764-3fb1293eaee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:35.007877 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830476c9-c5fd-4cab-a764-3fb1293eaee6 admin admin] PATH_INFO: `/v3/users/6d2b603f28e24598ab14e11993984d6f` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:35.016236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830476c9-c5fd-4cab-a764-3fb1293eaee6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d2b603f28e24598ab14e11993984d6f)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:35.016236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830476c9-c5fd-4cab-a764-3fb1293eaee6 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:35.068422 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0ef5a006-f2de-4477-baa4-8fc1ff6ecd3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a15f7edd4a454fd9a3e2060f00cd8c88'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:35.085051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-830476c9-c5fd-4cab-a764-3fb1293eaee6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d2b603f28e24598ab14e11993984d6f'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:35.085918 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-830476c9-c5fd-4cab-a764-3fb1293eaee6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d2b603f28e24598ab14e11993984d6f'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:35.100300 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0ef5a006-f2de-4477-baa4-8fc1ff6ecd3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a15f7edd4a454fd9a3e2060f00cd8c88'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:35.101098 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0ef5a006-f2de-4477-baa4-8fc1ff6ecd3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a15f7edd4a454fd9a3e2060f00cd8c88'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:35.105377 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-830476c9-c5fd-4cab-a764-3fb1293eaee6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d2b603f28e24598ab14e11993984d6f'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:35.115201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-830476c9-c5fd-4cab-a764-3fb1293eaee6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d2b603f28e24598ab14e11993984d6f'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:35.115994 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-830476c9-c5fd-4cab-a764-3fb1293eaee6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d2b603f28e24598ab14e11993984d6f'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:35.133118 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1112/2236] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:40:34 2022] DELETE /identity/v3/users/6d2b603f28e24598ab14e11993984d6f => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:40:35.137738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0ef5a006-f2de-4477-baa4-8fc1ff6ecd3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a15f7edd4a454fd9a3e2060f00cd8c88'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:35.138209 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d395baf-2d0b-4f97-bcbb-471a353f9369 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:35.138496 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d395baf-2d0b-4f97-bcbb-471a353f9369 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:35.138644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0ef5a006-f2de-4477-baa4-8fc1ff6ecd3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a15f7edd4a454fd9a3e2060f00cd8c88'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:35.139055 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d395baf-2d0b-4f97-bcbb-471a353f9369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:35.156098 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1125/2237] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:40:34 2022] DELETE /identity/v3/users/a15f7edd4a454fd9a3e2060f00cd8c88 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:40:35.161957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-6d395baf-2d0b-4f97-bcbb-471a353f9369 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:35.164904 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-6d395baf-2d0b-4f97-bcbb-471a353f9369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:35.264660 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1113/2238] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:40:35.300469 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5a006-f2de-4477-baa4-8fc1ff6ecd3a admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:35.302620 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0ef5a006-f2de-4477-baa4-8fc1ff6ecd3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:35.396463 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5a006-f2de-4477-baa4-8fc1ff6ecd3a admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:35.414379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e9d9a-4c05-4851-8904-215d79d21594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:35.415031 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e9d9a-4c05-4851-8904-215d79d21594 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:35.415310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e9d9a-4c05-4851-8904-215d79d21594 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:35.416118 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e9d9a-4c05-4851-8904-215d79d21594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:35.432190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e9d9a-4c05-4851-8904-215d79d21594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:35.433625 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e9d9a-4c05-4851-8904-215d79d21594 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:35.454474 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1126/2239] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:40:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:40:35.781018 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d395baf-2d0b-4f97-bcbb-471a353f9369 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:35.783026 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-6d395baf-2d0b-4f97-bcbb-471a353f9369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:35.797281 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d395baf-2d0b-4f97-bcbb-471a353f9369 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:35.802749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0288d852-b6ed-406d-b23e-6b3b7816bb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:35.803584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288d852-b6ed-406d-b23e-6b3b7816bb02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:35.803967 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288d852-b6ed-406d-b23e-6b3b7816bb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:35.805230 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288d852-b6ed-406d-b23e-6b3b7816bb02 admin admin] PATH_INFO: `/v3/projects/abb5d463425140969f0ae57e469aa36c` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:35.810390 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0288d852-b6ed-406d-b23e-6b3b7816bb02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abb5d463425140969f0ae57e469aa36c)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:35.810390 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0288d852-b6ed-406d-b23e-6b3b7816bb02 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:35.847266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88297b43-bd40-44d5-a8ee-905f7110a39b None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:35.847266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88297b43-bd40-44d5-a8ee-905f7110a39b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:35.847605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88297b43-bd40-44d5-a8ee-905f7110a39b None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:35.849126 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1127/2240] 10.222.0.47 () {54 vars in 921 bytes} [Thu Aug 4 04:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 04:40:35.879427 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:35.884127 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bb2b2-2f36-4fc9-a42b-5a7a360f10d6 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:35.884793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bb2b2-2f36-4fc9-a42b-5a7a360f10d6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:35.884937 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bb2b2-2f36-4fc9-a42b-5a7a360f10d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:35.913939 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-0288d852-b6ed-406d-b23e-6b3b7816bb02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abb5d463425140969f0ae57e469aa36c', 'request_id': 'req-0288d852-b6ed-406d-b23e-6b3b7816bb02'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:35.925142 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-0288d852-b6ed-406d-b23e-6b3b7816bb02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abb5d463425140969f0ae57e469aa36c', 'request_id': 'req-0288d852-b6ed-406d-b23e-6b3b7816bb02'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:35.926023 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-d30bb2b2-2f36-4fc9-a42b-5a7a360f10d6 None None] MFA Rules not processed for user `350f72f740f749f68175c1c4e8ae9d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:35.927486 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d30bb2b2-2f36-4fc9-a42b-5a7a360f10d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:35.934527 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-0288d852-b6ed-406d-b23e-6b3b7816bb02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abb5d463425140969f0ae57e469aa36c', 'request_id': 'req-0288d852-b6ed-406d-b23e-6b3b7816bb02'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:35.938992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:35.956218 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-0288d852-b6ed-406d-b23e-6b3b7816bb02 admin admin] The token cache is being invalidate because project abb5d463425140969f0ae57e469aa36c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:40:35.956660 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-0288d852-b6ed-406d-b23e-6b3b7816bb02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:35.957494 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-0288d852-b6ed-406d-b23e-6b3b7816bb02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:35.959455 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1114/2241] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:40:35 2022] DELETE /identity/v3/projects/abb5d463425140969f0ae57e469aa36c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:35.967163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bdba2-a36a-49db-87e5-b50adcb7bd23 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:35.967808 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bdba2-a36a-49db-87e5-b50adcb7bd23 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:35.968409 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bdba2-a36a-49db-87e5-b50adcb7bd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:36.012974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-1e6bdba2-a36a-49db-87e5-b50adcb7bd23 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:36.015085 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-1e6bdba2-a36a-49db-87e5-b50adcb7bd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:36.050770 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1128/2242] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Aug 4 04:40:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:40:36.053206 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30bb2b2-2f36-4fc9-a42b-5a7a360f10d6 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:36.054742 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d30bb2b2-2f36-4fc9-a42b-5a7a360f10d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:36.110093 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1115/2243] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:36.116502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bdba2-a36a-49db-87e5-b50adcb7bd23 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:36.116502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-1e6bdba2-a36a-49db-87e5-b50adcb7bd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:36.125105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bdba2-a36a-49db-87e5-b50adcb7bd23 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:36.129929 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04cc5d-a3bb-48fb-8d63-8c024c9455aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:36.130654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04cc5d-a3bb-48fb-8d63-8c024c9455aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:36.131000 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04cc5d-a3bb-48fb-8d63-8c024c9455aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:36.131288 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04cc5d-a3bb-48fb-8d63-8c024c9455aa admin admin] PATH_INFO: `/v3/users/35f3d17087dc4748b241350e76034ce8` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:36.132637 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04cc5d-a3bb-48fb-8d63-8c024c9455aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=35f3d17087dc4748b241350e76034ce8)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:36.134710 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04cc5d-a3bb-48fb-8d63-8c024c9455aa admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:36.149578 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30bb2b2-2f36-4fc9-a42b-5a7a360f10d6 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:36.155611 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547998a-e0a2-4f17-838f-a07ff504943a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:36.156412 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547998a-e0a2-4f17-838f-a07ff504943a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:36.156855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547998a-e0a2-4f17-838f-a07ff504943a admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:36.157202 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547998a-e0a2-4f17-838f-a07ff504943a admin admin] PATH_INFO: `/v3/projects/46680cd77cea47f381ea6e71e0d26fcb` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:36.158601 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d547998a-e0a2-4f17-838f-a07ff504943a admin admin] RBAC: Authorizing `identity:delete_project(project_id=46680cd77cea47f381ea6e71e0d26fcb)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:36.160543 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d547998a-e0a2-4f17-838f-a07ff504943a admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:36.193237 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-4a04cc5d-a3bb-48fb-8d63-8c024c9455aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35f3d17087dc4748b241350e76034ce8'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:36.193966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-4a04cc5d-a3bb-48fb-8d63-8c024c9455aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35f3d17087dc4748b241350e76034ce8'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:36.214271 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-4a04cc5d-a3bb-48fb-8d63-8c024c9455aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35f3d17087dc4748b241350e76034ce8'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:36.223953 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-4a04cc5d-a3bb-48fb-8d63-8c024c9455aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35f3d17087dc4748b241350e76034ce8'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:36.225072 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-4a04cc5d-a3bb-48fb-8d63-8c024c9455aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35f3d17087dc4748b241350e76034ce8'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:36.226982 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d547998a-e0a2-4f17-838f-a07ff504943a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46680cd77cea47f381ea6e71e0d26fcb', 'request_id': 'req-d547998a-e0a2-4f17-838f-a07ff504943a'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:36.236528 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d547998a-e0a2-4f17-838f-a07ff504943a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46680cd77cea47f381ea6e71e0d26fcb', 'request_id': 'req-d547998a-e0a2-4f17-838f-a07ff504943a'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:36.239608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1116/2244] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:40:36 2022] DELETE /identity/v3/users/35f3d17087dc4748b241350e76034ce8 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:36.240644 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04cc5d-a3bb-48fb-8d63-8c024c9455aa admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:36.242204 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-4a04cc5d-a3bb-48fb-8d63-8c024c9455aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:36.246254 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d547998a-e0a2-4f17-838f-a07ff504943a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46680cd77cea47f381ea6e71e0d26fcb', 'request_id': 'req-d547998a-e0a2-4f17-838f-a07ff504943a'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:36.262786 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d547998a-e0a2-4f17-838f-a07ff504943a admin admin] The token cache is being invalidate because project 46680cd77cea47f381ea6e71e0d26fcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:40:36.262786 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d547998a-e0a2-4f17-838f-a07ff504943a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:36.263256 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d547998a-e0a2-4f17-838f-a07ff504943a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:36.268899 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1129/2245] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:40:36 2022] DELETE /identity/v3/projects/46680cd77cea47f381ea6e71e0d26fcb => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:36.268899 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d95b28-9bcf-4a57-8260-353a7f269100 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:36.268899 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d95b28-9bcf-4a57-8260-353a7f269100 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:36.268899 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d95b28-9bcf-4a57-8260-353a7f269100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:36.272908 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:36.295958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-63d95b28-9bcf-4a57-8260-353a7f269100 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:36.297363 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-63d95b28-9bcf-4a57-8260-353a7f269100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:36.342322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04cc5d-a3bb-48fb-8d63-8c024c9455aa admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:36.347416 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:36.352138 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4bfcec-4c55-439b-966a-b9cab981d048 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350f72f740f749f68175c1c4e8ae9d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:36.355308 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4bfcec-4c55-439b-966a-b9cab981d048 service placement] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:36.355719 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4bfcec-4c55-439b-966a-b9cab981d048 service placement] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:36.356114 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4bfcec-4c55-439b-966a-b9cab981d048 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:36.360318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-7b4bfcec-4c55-439b-966a-b9cab981d048 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:36.392980 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4bfcec-4c55-439b-966a-b9cab981d048 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:36.395070 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4bfcec-4c55-439b-966a-b9cab981d048 service placement] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:36.405534 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1130/2246] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:36.406974 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aebc2f-7517-4385-88f3-a15d84581387 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:36.407360 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aebc2f-7517-4385-88f3-a15d84581387 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:36.407736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aebc2f-7517-4385-88f3-a15d84581387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:36.409023 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1117/2247] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:40:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:40:36.410553 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c082c5-e774-4409-b4d6-f0e83aa7d2bc None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:36.410849 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c082c5-e774-4409-b4d6-f0e83aa7d2bc None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:36.411129 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c082c5-e774-4409-b4d6-f0e83aa7d2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:36.430931 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-e8aebc2f-7517-4385-88f3-a15d84581387 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:36.433047 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e8aebc2f-7517-4385-88f3-a15d84581387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:36.439622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-c4c082c5-e774-4409-b4d6-f0e83aa7d2bc None None] MFA Rules not processed for user `1b23dd1311554910a19399aa74df9e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:36.441315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c4c082c5-e774-4409-b4d6-f0e83aa7d2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:36.445958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1131/2248] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:36.454780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aebc2f-7517-4385-88f3-a15d84581387 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:36.467774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aebc2f-7517-4385-88f3-a15d84581387 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:36.472978 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b825ddb-556a-4d75-81f3-318d264e2585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:36.473780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b825ddb-556a-4d75-81f3-318d264e2585 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:36.474057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b825ddb-556a-4d75-81f3-318d264e2585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:36.474331 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b825ddb-556a-4d75-81f3-318d264e2585 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:36.475092 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b825ddb-556a-4d75-81f3-318d264e2585 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:40:36.475400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b825ddb-556a-4d75-81f3-318d264e2585 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:36.477189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b825ddb-556a-4d75-81f3-318d264e2585 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:36.491447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1132/2249] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:40:36.497063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b274f38-8d30-4874-b900-239937a29b7c None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:36.497356 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b274f38-8d30-4874-b900-239937a29b7c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:36.497661 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b274f38-8d30-4874-b900-239937a29b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:36.531086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1118/2250] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:36.532103 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c082c5-e774-4409-b4d6-f0e83aa7d2bc None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:36.533613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c4c082c5-e774-4409-b4d6-f0e83aa7d2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:36.546416 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-1b274f38-8d30-4874-b900-239937a29b7c None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:36.547692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1b274f38-8d30-4874-b900-239937a29b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:36.552820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c082c5-e774-4409-b4d6-f0e83aa7d2bc None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:36.556539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1133/2251] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:36.559905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b274f38-8d30-4874-b900-239937a29b7c None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:36.560035 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8603920-2bd3-4537-bf20-157711059e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:36.560035 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8603920-2bd3-4537-bf20-157711059e77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:36.560456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8603920-2bd3-4537-bf20-157711059e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:36.560951 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8603920-2bd3-4537-bf20-157711059e77 admin admin] PATH_INFO: `/v3/projects/04e96b6523d9418997d83e1e57a4425e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:36.562972 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8603920-2bd3-4537-bf20-157711059e77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04e96b6523d9418997d83e1e57a4425e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:36.566297 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8603920-2bd3-4537-bf20-157711059e77 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:36.574408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b274f38-8d30-4874-b900-239937a29b7c None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:36.581190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe70f4-7e01-4279-b309-a1fba7f6a644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:36.581756 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe70f4-7e01-4279-b309-a1fba7f6a644 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:36.582044 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe70f4-7e01-4279-b309-a1fba7f6a644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:36.582611 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe70f4-7e01-4279-b309-a1fba7f6a644 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:36.582998 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fe70f4-7e01-4279-b309-a1fba7f6a644 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:36.585172 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fe70f4-7e01-4279-b309-a1fba7f6a644 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:36.632563 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-f8603920-2bd3-4537-bf20-157711059e77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04e96b6523d9418997d83e1e57a4425e', 'request_id': 'req-f8603920-2bd3-4537-bf20-157711059e77'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:36.636658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1134/2252] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:40:36 2022] POST /identity/v3/projects => generated 363 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:40:36.640473 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe70f4-7e01-4279-b309-a1fba7f6a644 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:36.646465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-f8603920-2bd3-4537-bf20-157711059e77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04e96b6523d9418997d83e1e57a4425e', 'request_id': 'req-f8603920-2bd3-4537-bf20-157711059e77'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:36.655054 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-f8603920-2bd3-4537-bf20-157711059e77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04e96b6523d9418997d83e1e57a4425e', 'request_id': 'req-f8603920-2bd3-4537-bf20-157711059e77'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:36.669885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-f8603920-2bd3-4537-bf20-157711059e77 admin admin] The token cache is being invalidate because project 04e96b6523d9418997d83e1e57a4425e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:40:36.670243 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-f8603920-2bd3-4537-bf20-157711059e77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:36.671214 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-f8603920-2bd3-4537-bf20-157711059e77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:36.676388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1119/2253] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:40:36 2022] DELETE /identity/v3/projects/04e96b6523d9418997d83e1e57a4425e => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:36.676388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8603920-2bd3-4537-bf20-157711059e77 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:36.676388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-f8603920-2bd3-4537-bf20-157711059e77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:36.734655 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe70f4-7e01-4279-b309-a1fba7f6a644 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:36.738902 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:36.742001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d9ebb-dd2d-4f82-9988-cce2d9fe6ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:36.743102 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d9ebb-dd2d-4f82-9988-cce2d9fe6ccd admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:36.743381 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d9ebb-dd2d-4f82-9988-cce2d9fe6ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:36.743701 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d9ebb-dd2d-4f82-9988-cce2d9fe6ccd admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:36.744440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d9ebb-dd2d-4f82-9988-cce2d9fe6ccd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:40:36.744745 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d9ebb-dd2d-4f82-9988-cce2d9fe6ccd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:36.753159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d9ebb-dd2d-4f82-9988-cce2d9fe6ccd admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:36.772682 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1135/2254] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:40:36.772682 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d9ebb-dd2d-4f82-9988-cce2d9fe6ccd admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:36.776963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4a8d9ebb-dd2d-4f82-9988-cce2d9fe6ccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:36.779719 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8603920-2bd3-4537-bf20-157711059e77 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:36.793406 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50af36-f04f-48da-91fa-9004caea7507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:36.794215 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50af36-f04f-48da-91fa-9004caea7507 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:36.794636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50af36-f04f-48da-91fa-9004caea7507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:36.794972 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50af36-f04f-48da-91fa-9004caea7507 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:36.795728 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50af36-f04f-48da-91fa-9004caea7507 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:36.798469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50af36-f04f-48da-91fa-9004caea7507 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:36.800020 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d9ebb-dd2d-4f82-9988-cce2d9fe6ccd admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:36.805705 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c17cad-0104-455c-883e-97b5788691fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:36.806870 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c17cad-0104-455c-883e-97b5788691fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:36.807151 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c17cad-0104-455c-883e-97b5788691fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:36.807435 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c17cad-0104-455c-883e-97b5788691fc admin admin] PATH_INFO: `/v3/users/342740bcb7c44209978dfd38dc8de785` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:36.808974 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c17cad-0104-455c-883e-97b5788691fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=342740bcb7c44209978dfd38dc8de785)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:36.812838 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c17cad-0104-455c-883e-97b5788691fc admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:36.840273 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1120/2255] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:40:36 2022] POST /identity/v3/users => generated 282 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:40:36.840273 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50af36-f04f-48da-91fa-9004caea7507 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:36.842178 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-7c50af36-f04f-48da-91fa-9004caea7507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:36.858043 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50af36-f04f-48da-91fa-9004caea7507 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:36.862738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56cc78-2d98-480c-ab94-e128f416ad5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:36.863414 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56cc78-2d98-480c-ab94-e128f416ad5a admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:36.863754 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56cc78-2d98-480c-ab94-e128f416ad5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:36.864071 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56cc78-2d98-480c-ab94-e128f416ad5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:36.864945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f56cc78-2d98-480c-ab94-e128f416ad5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:36.867237 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f56cc78-2d98-480c-ab94-e128f416ad5a admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:36.891657 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-99c17cad-0104-455c-883e-97b5788691fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '342740bcb7c44209978dfd38dc8de785'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:36.899342 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-99c17cad-0104-455c-883e-97b5788691fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '342740bcb7c44209978dfd38dc8de785'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:36.900926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-99c17cad-0104-455c-883e-97b5788691fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '342740bcb7c44209978dfd38dc8de785'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:36.912808 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1121/2256] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:40:36 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:40:36.914471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56cc78-2d98-480c-ab94-e128f416ad5a admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:36.917906 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-6f56cc78-2d98-480c-ab94-e128f416ad5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:36.919407 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-99c17cad-0104-455c-883e-97b5788691fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '342740bcb7c44209978dfd38dc8de785'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:36.920412 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-99c17cad-0104-455c-883e-97b5788691fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '342740bcb7c44209978dfd38dc8de785'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:36.931100 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:36.938106 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1136/2257] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:40:36 2022] DELETE /identity/v3/users/342740bcb7c44209978dfd38dc8de785 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:36.952972 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c17cad-0104-455c-883e-97b5788691fc admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:36.955815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-99c17cad-0104-455c-883e-97b5788691fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:37.032214 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56cc78-2d98-480c-ab94-e128f416ad5a admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:37.134103 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c17cad-0104-455c-883e-97b5788691fc admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:37.139501 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d6c05-1af2-4045-8b23-e27dfbd80e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:37.140233 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d6c05-1af2-4045-8b23-e27dfbd80e39 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:37.140524 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d6c05-1af2-4045-8b23-e27dfbd80e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:37.140798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d6c05-1af2-4045-8b23-e27dfbd80e39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:37.141591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d6c05-1af2-4045-8b23-e27dfbd80e39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:37.143146 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d6c05-1af2-4045-8b23-e27dfbd80e39 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:37.155602 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753644b-1cda-49a5-84db-4bc82a430906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:37.159843 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753644b-1cda-49a5-84db-4bc82a430906 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:37.160161 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753644b-1cda-49a5-84db-4bc82a430906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:37.160406 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753644b-1cda-49a5-84db-4bc82a430906 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:37.161256 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2753644b-1cda-49a5-84db-4bc82a430906 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:37.167145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2753644b-1cda-49a5-84db-4bc82a430906 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:37.172940 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1122/2258] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:37.177296 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d6c05-1af2-4045-8b23-e27dfbd80e39 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:37.203719 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d6c05-1af2-4045-8b23-e27dfbd80e39 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:37.208665 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1137/2259] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:40:36 2022] POST /identity/v3/users => generated 293 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:40:37.212997 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa84f7-17c7-4607-9167-4b07e44b245e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:37.214503 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753644b-1cda-49a5-84db-4bc82a430906 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:37.214626 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa84f7-17c7-4607-9167-4b07e44b245e admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:37.214626 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa84f7-17c7-4607-9167-4b07e44b245e admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:37.215339 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa84f7-17c7-4607-9167-4b07e44b245e admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:37.216587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa84f7-17c7-4607-9167-4b07e44b245e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:37.220925 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa84f7-17c7-4607-9167-4b07e44b245e admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:37.231438 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753644b-1cda-49a5-84db-4bc82a430906 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:37.241970 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1123/2260] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:37.243006 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa84f7-17c7-4607-9167-4b07e44b245e admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:37.245431 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-f8aa84f7-17c7-4607-9167-4b07e44b245e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:37.247728 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e72e84-d2fd-4650-a982-f8d2e52cc2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:37.248383 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e72e84-d2fd-4650-a982-f8d2e52cc2a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:37.248670 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e72e84-d2fd-4650-a982-f8d2e52cc2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:37.248947 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e72e84-d2fd-4650-a982-f8d2e52cc2a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:37.249698 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e72e84-d2fd-4650-a982-f8d2e52cc2a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:37.252129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e72e84-d2fd-4650-a982-f8d2e52cc2a3 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:37.270915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa84f7-17c7-4607-9167-4b07e44b245e admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:37.276712 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5e73d-c656-45d4-9d9d-651d2cccd7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:37.277423 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5e73d-c656-45d4-9d9d-651d2cccd7c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:37.277735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5e73d-c656-45d4-9d9d-651d2cccd7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:37.277918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1138/2261] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 04:40:37.278818 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e72e84-d2fd-4650-a982-f8d2e52cc2a3 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:37.285659 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5e73d-c656-45d4-9d9d-651d2cccd7c2 admin admin] PATH_INFO: `/v3/projects/6308adc367df42b9805d3556a5a95b6b` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:37.287138 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb5e73d-c656-45d4-9d9d-651d2cccd7c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6308adc367df42b9805d3556a5a95b6b)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:37.290002 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb5e73d-c656-45d4-9d9d-651d2cccd7c2 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:37.302261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e72e84-d2fd-4650-a982-f8d2e52cc2a3 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:37.307575 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86d31b-0655-4e2a-9dc1-7bfc907a7ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:37.308280 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86d31b-0655-4e2a-9dc1-7bfc907a7ca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:37.308463 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86d31b-0655-4e2a-9dc1-7bfc907a7ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:37.308706 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86d31b-0655-4e2a-9dc1-7bfc907a7ca2 admin admin] PATH_INFO: `/v3/projects/b2ce617c0ceb49c3ba0e23d5d315a769/users/e6b9ba60aaf8497a9fcd941d8308b11d/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:37.327667 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86d31b-0655-4e2a-9dc1-7bfc907a7ca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2ce617c0ceb49c3ba0e23d5d315a769, user_id=e6b9ba60aaf8497a9fcd941d8308b11d, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:37.331057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86d31b-0655-4e2a-9dc1-7bfc907a7ca2 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:37.369625 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1139/2262] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:40:37 2022] PUT /identity/v3/projects/b2ce617c0ceb49c3ba0e23d5d315a769/users/e6b9ba60aaf8497a9fcd941d8308b11d/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:37.369625 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86d31b-0655-4e2a-9dc1-7bfc907a7ca2 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:37.378767 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:37.400565 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-2fb5e73d-c656-45d4-9d9d-651d2cccd7c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6308adc367df42b9805d3556a5a95b6b', 'request_id': 'req-2fb5e73d-c656-45d4-9d9d-651d2cccd7c2'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:37.424487 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-2fb5e73d-c656-45d4-9d9d-651d2cccd7c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6308adc367df42b9805d3556a5a95b6b', 'request_id': 'req-2fb5e73d-c656-45d4-9d9d-651d2cccd7c2'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:37.444031 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-2fb5e73d-c656-45d4-9d9d-651d2cccd7c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6308adc367df42b9805d3556a5a95b6b', 'request_id': 'req-2fb5e73d-c656-45d4-9d9d-651d2cccd7c2'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:37.460139 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-2fb5e73d-c656-45d4-9d9d-651d2cccd7c2 admin admin] The token cache is being invalidate because project 6308adc367df42b9805d3556a5a95b6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:40:37.460509 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-2fb5e73d-c656-45d4-9d9d-651d2cccd7c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:37.461187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-2fb5e73d-c656-45d4-9d9d-651d2cccd7c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:37.466602 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1124/2263] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:40:37 2022] DELETE /identity/v3/projects/6308adc367df42b9805d3556a5a95b6b => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:37.468153 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd27244-6886-4e1d-b78a-43d0d5908f34 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:37.468446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd27244-6886-4e1d-b78a-43d0d5908f34 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:37.468717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd27244-6886-4e1d-b78a-43d0d5908f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:37.521575 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86d31b-0655-4e2a-9dc1-7bfc907a7ca2 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:37.554477 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-6dd27244-6886-4e1d-b78a-43d0d5908f34 None None] MFA Rules not processed for user `e6b9ba60aaf8497a9fcd941d8308b11d`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:37.558108 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-6dd27244-6886-4e1d-b78a-43d0d5908f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:37.673646 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c461b1-a7f8-4599-bc2b-580f0e9ae8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:37.674253 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c461b1-a7f8-4599-bc2b-580f0e9ae8e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:37.674598 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c461b1-a7f8-4599-bc2b-580f0e9ae8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:37.674947 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c461b1-a7f8-4599-bc2b-580f0e9ae8e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:37.675682 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c461b1-a7f8-4599-bc2b-580f0e9ae8e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:37.677348 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c461b1-a7f8-4599-bc2b-580f0e9ae8e3 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:37.679354 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:37.708276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1140/2264] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:37.717172 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4d5ab-115c-4378-9f1d-6c7648210914 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:37.717496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4d5ab-115c-4378-9f1d-6c7648210914 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:37.717771 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4d5ab-115c-4378-9f1d-6c7648210914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:37.730831 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1125/2265] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:37.734563 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd27244-6886-4e1d-b78a-43d0d5908f34 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:37.738155 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-6dd27244-6886-4e1d-b78a-43d0d5908f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:37.774094 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-bce4d5ab-115c-4378-9f1d-6c7648210914 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:37.774310 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd27244-6886-4e1d-b78a-43d0d5908f34 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:37.776798 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-bce4d5ab-115c-4378-9f1d-6c7648210914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:37.782931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f6d6b-109b-4a05-aa44-014e66d153d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:37.783686 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f6d6b-109b-4a05-aa44-014e66d153d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:37.783972 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f6d6b-109b-4a05-aa44-014e66d153d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:37.784379 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f6d6b-109b-4a05-aa44-014e66d153d0 admin admin] PATH_INFO: `/v3/projects/bd7fc43776684ea99eb660966ed97695/users/2949f24248034487ada1e11bc9a9e4d7/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:37.809008 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1141/2266] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:37.810293 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4d5ab-115c-4378-9f1d-6c7648210914 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:37.813525 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-bce4d5ab-115c-4378-9f1d-6c7648210914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:37.821209 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f6d6b-109b-4a05-aa44-014e66d153d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd7fc43776684ea99eb660966ed97695, user_id=2949f24248034487ada1e11bc9a9e4d7, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:37.828658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f6d6b-109b-4a05-aa44-014e66d153d0 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:37.848707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4d5ab-115c-4378-9f1d-6c7648210914 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:37.862890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1126/2267] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:40:37 2022] PUT /identity/v3/projects/bd7fc43776684ea99eb660966ed97695/users/2949f24248034487ada1e11bc9a9e4d7/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:37.867498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbd9f3-1fee-48f7-9302-3e9a5e4aa645 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:37.867840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbd9f3-1fee-48f7-9302-3e9a5e4aa645 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:37.868119 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbd9f3-1fee-48f7-9302-3e9a5e4aa645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:37.897615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-5efbd9f3-1fee-48f7-9302-3e9a5e4aa645 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:37.898736 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5efbd9f3-1fee-48f7-9302-3e9a5e4aa645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:37.934632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04bd953-8979-4893-a114-eb2c6e8285e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:37.935769 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bd953-8979-4893-a114-eb2c6e8285e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:37.935769 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bd953-8979-4893-a114-eb2c6e8285e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:37.936065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bd953-8979-4893-a114-eb2c6e8285e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:37.936674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04bd953-8979-4893-a114-eb2c6e8285e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:37.938691 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04bd953-8979-4893-a114-eb2c6e8285e0 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:37.981484 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1127/2268] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:37.983021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a990b97-3a7e-4bf5-ab92-17c91a5c6994 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:37.983318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a990b97-3a7e-4bf5-ab92-17c91a5c6994 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:37.983587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a990b97-3a7e-4bf5-ab92-17c91a5c6994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:37.986037 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1142/2269] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:40:37 2022] POST /identity/v3/projects => generated 365 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:40:37.989580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04bd953-8979-4893-a114-eb2c6e8285e0 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:38.011499 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-5a990b97-3a7e-4bf5-ab92-17c91a5c6994 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:38.012854 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5a990b97-3a7e-4bf5-ab92-17c91a5c6994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:38.062186 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:38.091040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04bd953-8979-4893-a114-eb2c6e8285e0 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:38.098550 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68986bd0-323c-4f64-a612-11e566bed2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:38.099225 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68986bd0-323c-4f64-a612-11e566bed2f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:38.099508 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68986bd0-323c-4f64-a612-11e566bed2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:38.099783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68986bd0-323c-4f64-a612-11e566bed2f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:38.100503 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68986bd0-323c-4f64-a612-11e566bed2f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:40:38.100810 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68986bd0-323c-4f64-a612-11e566bed2f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:38.102465 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68986bd0-323c-4f64-a612-11e566bed2f1 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:38.121601 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1143/2270] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:40:38.121601 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68986bd0-323c-4f64-a612-11e566bed2f1 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:38.126007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1128/2271] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:38.127234 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a990b97-3a7e-4bf5-ab92-17c91a5c6994 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:38.139149 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a990b97-3a7e-4bf5-ab92-17c91a5c6994 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:38.140731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68986bd0-323c-4f64-a612-11e566bed2f1 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:38.145734 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52118a87-5a00-46bd-95a8-7ec31411d186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:38.146359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52118a87-5a00-46bd-95a8-7ec31411d186 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:38.146635 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52118a87-5a00-46bd-95a8-7ec31411d186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:38.146908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52118a87-5a00-46bd-95a8-7ec31411d186 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:38.147572 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52118a87-5a00-46bd-95a8-7ec31411d186 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:38.150456 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52118a87-5a00-46bd-95a8-7ec31411d186 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:38.152522 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46291b0a-541e-4f04-b062-be859a4778bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:38.154653 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46291b0a-541e-4f04-b062-be859a4778bf admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:38.155765 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46291b0a-541e-4f04-b062-be859a4778bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:38.156241 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46291b0a-541e-4f04-b062-be859a4778bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:38.158018 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46291b0a-541e-4f04-b062-be859a4778bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:38.162601 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46291b0a-541e-4f04-b062-be859a4778bf admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:38.185845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1144/2272] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:40:38 2022] POST /identity/v3/users => generated 289 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:40:38.186821 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52118a87-5a00-46bd-95a8-7ec31411d186 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:38.190522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-52118a87-5a00-46bd-95a8-7ec31411d186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:38.213225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1129/2273] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:40:38 2022] POST /identity/v3/projects => generated 385 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:40:38.217363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46291b0a-541e-4f04-b062-be859a4778bf admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:38.312335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52118a87-5a00-46bd-95a8-7ec31411d186 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:38.320969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:38.374275 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46291b0a-541e-4f04-b062-be859a4778bf admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:38.396784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0736fbb-0c33-47e0-ba08-3d744ac28c9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:38.397782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0736fbb-0c33-47e0-ba08-3d744ac28c9d service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:38.397782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0736fbb-0c33-47e0-ba08-3d744ac28c9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:38.398577 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0736fbb-0c33-47e0-ba08-3d744ac28c9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:38.402800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a0123-77b9-4425-ac2f-71233ccedd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:38.403793 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a0123-77b9-4425-ac2f-71233ccedd64 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:38.403925 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a0123-77b9-4425-ac2f-71233ccedd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:38.404131 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a0123-77b9-4425-ac2f-71233ccedd64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:38.404858 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a0123-77b9-4425-ac2f-71233ccedd64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:38.409916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0736fbb-0c33-47e0-ba08-3d744ac28c9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:38.411435 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0736fbb-0c33-47e0-ba08-3d744ac28c9d service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:38.424712 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a0123-77b9-4425-ac2f-71233ccedd64 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:38.436101 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1145/2274] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:40:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:40:38.437486 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0736fbb-0c33-47e0-ba08-3d744ac28c9d service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:38.461672 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1130/2275] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:38.461994 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0736fbb-0c33-47e0-ba08-3d744ac28c9d service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:38.463338 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a0123-77b9-4425-ac2f-71233ccedd64 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:38.480732 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be818198-d920-4648-9671-8e121df76195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:38.481755 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be818198-d920-4648-9671-8e121df76195 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:38.482035 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be818198-d920-4648-9671-8e121df76195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:38.482310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be818198-d920-4648-9671-8e121df76195 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:38.483236 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be818198-d920-4648-9671-8e121df76195 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:38.486127 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be818198-d920-4648-9671-8e121df76195 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:38.489458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a0123-77b9-4425-ac2f-71233ccedd64 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:38.502825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0781b6-fa55-4a51-a497-348bf0bc81ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:38.505318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0781b6-fa55-4a51-a497-348bf0bc81ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:38.506603 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0781b6-fa55-4a51-a497-348bf0bc81ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:38.508876 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0781b6-fa55-4a51-a497-348bf0bc81ed admin admin] PATH_INFO: `/v3/projects/6a92c9c6efd24b9ea5ca3ca69320204a/users/38074f0063754bd7bfcca31d8170a941/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:38.544129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1146/2276] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:40:38 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:40:38.549472 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be818198-d920-4648-9671-8e121df76195 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:38.561314 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0781b6-fa55-4a51-a497-348bf0bc81ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a92c9c6efd24b9ea5ca3ca69320204a, user_id=38074f0063754bd7bfcca31d8170a941, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:38.572295 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0781b6-fa55-4a51-a497-348bf0bc81ed admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:38.575384 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be818198-d920-4648-9671-8e121df76195 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:38.583637 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ab295-16fc-4a3d-80b3-f233c4d44916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:38.584428 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ab295-16fc-4a3d-80b3-f233c4d44916 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:38.584775 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ab295-16fc-4a3d-80b3-f233c4d44916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:38.585158 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ab295-16fc-4a3d-80b3-f233c4d44916 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:38.585896 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ab295-16fc-4a3d-80b3-f233c4d44916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:38.588883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ab295-16fc-4a3d-80b3-f233c4d44916 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:38.601202 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1131/2277] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:40:38 2022] PUT /identity/v3/projects/6a92c9c6efd24b9ea5ca3ca69320204a/users/38074f0063754bd7bfcca31d8170a941/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:38.602383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0781b6-fa55-4a51-a497-348bf0bc81ed admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:38.607001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1147/2278] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:38.611338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ab295-16fc-4a3d-80b3-f233c4d44916 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:38.632697 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ab295-16fc-4a3d-80b3-f233c4d44916 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:38.782333 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32a6e5-098a-44a1-934d-a918b0b27535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:38.782978 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32a6e5-098a-44a1-934d-a918b0b27535 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:38.783975 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0781b6-fa55-4a51-a497-348bf0bc81ed admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:38.785030 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32a6e5-098a-44a1-934d-a918b0b27535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:38.785312 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32a6e5-098a-44a1-934d-a918b0b27535 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:38.787682 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32a6e5-098a-44a1-934d-a918b0b27535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:38.792169 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32a6e5-098a-44a1-934d-a918b0b27535 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:38.795468 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1919709-f8ff-4cab-8e75-033e78811635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:38.796870 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1919709-f8ff-4cab-8e75-033e78811635 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:38.797161 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1919709-f8ff-4cab-8e75-033e78811635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:38.797827 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1919709-f8ff-4cab-8e75-033e78811635 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:38.800481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1919709-f8ff-4cab-8e75-033e78811635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:38.804292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1919709-f8ff-4cab-8e75-033e78811635 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:38.816325 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1148/2279] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:38.820572 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32a6e5-098a-44a1-934d-a918b0b27535 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:38.835035 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1132/2280] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:38.853942 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1919709-f8ff-4cab-8e75-033e78811635 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:38.854902 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32a6e5-098a-44a1-934d-a918b0b27535 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:38.877907 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c2305-c7c9-4151-b289-a9996182f782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:38.878888 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c2305-c7c9-4151-b289-a9996182f782 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:38.879896 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c2305-c7c9-4151-b289-a9996182f782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:38.880284 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c2305-c7c9-4151-b289-a9996182f782 admin admin] PATH_INFO: `/v3/projects/9606e6021ff44458900026939e58fdb4/users/52ec3d2aa0e347e3bf7dcfa68743b059/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:38.884534 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1919709-f8ff-4cab-8e75-033e78811635 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:38.891329 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b46d24-251e-4a22-a2a8-7e66e09691ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:38.892094 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b46d24-251e-4a22-a2a8-7e66e09691ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:38.892452 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b46d24-251e-4a22-a2a8-7e66e09691ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:38.892880 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b46d24-251e-4a22-a2a8-7e66e09691ff admin admin] PATH_INFO: `/v3/projects/6a92c9c6efd24b9ea5ca3ca69320204a/users/38074f0063754bd7bfcca31d8170a941/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:38.895400 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b46d24-251e-4a22-a2a8-7e66e09691ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a92c9c6efd24b9ea5ca3ca69320204a, user_id=38074f0063754bd7bfcca31d8170a941, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:38.907628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b46d24-251e-4a22-a2a8-7e66e09691ff admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:38.932412 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c2305-c7c9-4151-b289-a9996182f782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9606e6021ff44458900026939e58fdb4, user_id=52ec3d2aa0e347e3bf7dcfa68743b059, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:38.937212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1133/2281] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:40:38 2022] PUT /identity/v3/projects/6a92c9c6efd24b9ea5ca3ca69320204a/users/38074f0063754bd7bfcca31d8170a941/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:38.947849 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c2305-c7c9-4151-b289-a9996182f782 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:38.951458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1246a7-59ba-4e10-9b23-e6261151bc9f None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:38.952154 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1246a7-59ba-4e10-9b23-e6261151bc9f None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:38.952618 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1246a7-59ba-4e10-9b23-e6261151bc9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:38.971564 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1149/2282] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:40:38 2022] PUT /identity/v3/projects/9606e6021ff44458900026939e58fdb4/users/52ec3d2aa0e347e3bf7dcfa68743b059/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:38.977400 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:38.988390 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf439d1-4c75-4b41-b854-1496fb5943bf None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:38.988732 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf439d1-4c75-4b41-b854-1496fb5943bf None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:38.989117 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf439d1-4c75-4b41-b854-1496fb5943bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:39.051612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-2c1246a7-59ba-4e10-9b23-e6261151bc9f None None] MFA Rules not processed for user `38074f0063754bd7bfcca31d8170a941`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:39.053117 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-2c1246a7-59ba-4e10-9b23-e6261151bc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:39.076242 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-bbf439d1-4c75-4b41-b854-1496fb5943bf None None] MFA Rules not processed for user `52ec3d2aa0e347e3bf7dcfa68743b059`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:39.078734 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-bbf439d1-4c75-4b41-b854-1496fb5943bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:39.190526 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:39.245459 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1150/2283] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:39.251475 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf439d1-4c75-4b41-b854-1496fb5943bf None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:39.300115 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1134/2284] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:38 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:39.305140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1246a7-59ba-4e10-9b23-e6261151bc9f None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:39.307655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-2c1246a7-59ba-4e10-9b23-e6261151bc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:39.395906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf439d1-4c75-4b41-b854-1496fb5943bf None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:39.405229 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1246a7-59ba-4e10-9b23-e6261151bc9f None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:39.410118 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3a5db-a0a0-44b4-a628-b09b61ccedba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:39.410768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3a5db-a0a0-44b4-a628-b09b61ccedba admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:39.411064 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3a5db-a0a0-44b4-a628-b09b61ccedba admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:39.411317 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74845ed0-c678-4e42-94c6-83f15579f32f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:39.411511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3a5db-a0a0-44b4-a628-b09b61ccedba admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:39.411923 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74845ed0-c678-4e42-94c6-83f15579f32f service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:39.412203 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74845ed0-c678-4e42-94c6-83f15579f32f service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:39.412524 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74845ed0-c678-4e42-94c6-83f15579f32f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:39.414160 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3a5db-a0a0-44b4-a628-b09b61ccedba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:39.417176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3a5db-a0a0-44b4-a628-b09b61ccedba admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:39.431864 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74845ed0-c678-4e42-94c6-83f15579f32f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:39.434751 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74845ed0-c678-4e42-94c6-83f15579f32f service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:39.464288 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1135/2285] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:40:39 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:40:39.476942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1151/2286] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:40:39 2022] POST /identity/v3/projects => generated 385 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:40:39.481718 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74845ed0-c678-4e42-94c6-83f15579f32f service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:39.496378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74845ed0-c678-4e42-94c6-83f15579f32f service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:39.581477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3a5db-a0a0-44b4-a628-b09b61ccedba admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:39.623809 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238125c8-66ea-45b6-a8b3-44ac74ff7600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:39.624997 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238125c8-66ea-45b6-a8b3-44ac74ff7600 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:39.625751 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238125c8-66ea-45b6-a8b3-44ac74ff7600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:39.626433 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238125c8-66ea-45b6-a8b3-44ac74ff7600 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:39.627488 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238125c8-66ea-45b6-a8b3-44ac74ff7600 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:39.630926 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238125c8-66ea-45b6-a8b3-44ac74ff7600 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:39.675434 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1136/2287] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:40:39 2022] POST /identity/v3/users => generated 299 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:40:39.685128 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238125c8-66ea-45b6-a8b3-44ac74ff7600 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:39.700997 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238125c8-66ea-45b6-a8b3-44ac74ff7600 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:39.707491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3a5db-a0a0-44b4-a628-b09b61ccedba admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:39.716475 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c6c3c-5e2d-415a-8052-4fb8409d46e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:39.717212 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c6c3c-5e2d-415a-8052-4fb8409d46e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:39.717585 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c6c3c-5e2d-415a-8052-4fb8409d46e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:39.718624 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c6c3c-5e2d-415a-8052-4fb8409d46e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:39.721814 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19422942-fe09-4b3c-bbae-bc03bd59be02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:39.723963 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19422942-fe09-4b3c-bbae-bc03bd59be02 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:39.724705 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19422942-fe09-4b3c-bbae-bc03bd59be02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:39.725695 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19422942-fe09-4b3c-bbae-bc03bd59be02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:39.727005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19422942-fe09-4b3c-bbae-bc03bd59be02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:39.731696 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19422942-fe09-4b3c-bbae-bc03bd59be02 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:39.752870 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1137/2288] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 04:40:39.757029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19422942-fe09-4b3c-bbae-bc03bd59be02 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:39.775237 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19422942-fe09-4b3c-bbae-bc03bd59be02 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:39.785660 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3ebd7-0181-4ace-880d-8bc1d2ae0229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:39.786711 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3ebd7-0181-4ace-880d-8bc1d2ae0229 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:39.787171 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3ebd7-0181-4ace-880d-8bc1d2ae0229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:39.798524 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3ebd7-0181-4ace-880d-8bc1d2ae0229 admin admin] PATH_INFO: `/v3/projects/146da10b6b6346f683aef5cc71f84a63/users/cdb2b6e419fc472abbb415b6b5917670/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:39.804632 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:39.847071 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3ebd7-0181-4ace-880d-8bc1d2ae0229 admin admin] RBAC: Authorizing `identity:create_grant(project_id=146da10b6b6346f683aef5cc71f84a63, user_id=cdb2b6e419fc472abbb415b6b5917670, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:39.847295 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00c6c3c-5e2d-415a-8052-4fb8409d46e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:39.851263 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00c6c3c-5e2d-415a-8052-4fb8409d46e5 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:39.853957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3ebd7-0181-4ace-880d-8bc1d2ae0229 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:39.867744 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1152/2289] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:40:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:40:39.889689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1138/2290] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:40:39 2022] PUT /identity/v3/projects/146da10b6b6346f683aef5cc71f84a63/users/cdb2b6e419fc472abbb415b6b5917670/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:39.891077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c6c3c-5e2d-415a-8052-4fb8409d46e5 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:39.930014 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c6c3c-5e2d-415a-8052-4fb8409d46e5 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:39.982765 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:40.103102 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a070a7-2e1e-46ec-af09-e771cef985a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:40.103754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a070a7-2e1e-46ec-af09-e771cef985a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:40.104041 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a070a7-2e1e-46ec-af09-e771cef985a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:40.106482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a070a7-2e1e-46ec-af09-e771cef985a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:40.109717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a070a7-2e1e-46ec-af09-e771cef985a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:40.113917 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a070a7-2e1e-46ec-af09-e771cef985a8 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:40.140328 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1153/2291] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:40.146171 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3ebd7-0181-4ace-880d-8bc1d2ae0229 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:40.186824 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3ebd7-0181-4ace-880d-8bc1d2ae0229 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:40.202891 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac0813-1f40-46d8-9edb-99464abc2d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:40.204655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac0813-1f40-46d8-9edb-99464abc2d59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:40.204959 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac0813-1f40-46d8-9edb-99464abc2d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:40.205240 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac0813-1f40-46d8-9edb-99464abc2d59 admin admin] PATH_INFO: `/v3/projects/146da10b6b6346f683aef5cc71f84a63/users/cdb2b6e419fc472abbb415b6b5917670/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:40.211849 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eac0813-1f40-46d8-9edb-99464abc2d59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=146da10b6b6346f683aef5cc71f84a63, user_id=cdb2b6e419fc472abbb415b6b5917670, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:40.218848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eac0813-1f40-46d8-9edb-99464abc2d59 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:40.291025 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1139/2292] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:40:40 2022] PUT /identity/v3/projects/146da10b6b6346f683aef5cc71f84a63/users/cdb2b6e419fc472abbb415b6b5917670/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:40.293323 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b76763-bed8-4217-a651-8c3f6e1d8a22 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:40.293645 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b76763-bed8-4217-a651-8c3f6e1d8a22 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:40.293930 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b76763-bed8-4217-a651-8c3f6e1d8a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:40.373348 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-a8b76763-bed8-4217-a651-8c3f6e1d8a22 None None] MFA Rules not processed for user `cdb2b6e419fc472abbb415b6b5917670`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:40.374839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a8b76763-bed8-4217-a651-8c3f6e1d8a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:40.576828 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1154/2293] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:40.620810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac0813-1f40-46d8-9edb-99464abc2d59 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:40.768004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac0813-1f40-46d8-9edb-99464abc2d59 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:40.774070 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3544b-bb89-476d-a2d3-4587c1741e20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:40.775782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3544b-bb89-476d-a2d3-4587c1741e20 service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:40.776275 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3544b-bb89-476d-a2d3-4587c1741e20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:40.777564 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3544b-bb89-476d-a2d3-4587c1741e20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:40.897830 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3544b-bb89-476d-a2d3-4587c1741e20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:40.900879 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3544b-bb89-476d-a2d3-4587c1741e20 service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:40.925417 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1140/2294] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:40:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:40:45.796445 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b76763-bed8-4217-a651-8c3f6e1d8a22 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:45.896178 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3544b-bb89-476d-a2d3-4587c1741e20 service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:45.907452 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3544b-bb89-476d-a2d3-4587c1741e20 service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:45.914127 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad466ae-bba1-4979-bfd3-6da699c64b04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:45.914768 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad466ae-bba1-4979-bfd3-6da699c64b04 service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:45.915785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad466ae-bba1-4979-bfd3-6da699c64b04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:45.916143 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad466ae-bba1-4979-bfd3-6da699c64b04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:45.919904 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b76763-bed8-4217-a651-8c3f6e1d8a22 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:45.924822 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532dcf3-eb2a-419e-a915-b17438ce94d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:45.925649 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532dcf3-eb2a-419e-a915-b17438ce94d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:45.925936 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532dcf3-eb2a-419e-a915-b17438ce94d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:45.926212 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532dcf3-eb2a-419e-a915-b17438ce94d4 admin admin] PATH_INFO: `/v3/users/e6b9ba60aaf8497a9fcd941d8308b11d` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:45.927630 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532dcf3-eb2a-419e-a915-b17438ce94d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6b9ba60aaf8497a9fcd941d8308b11d)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:45.929646 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532dcf3-eb2a-419e-a915-b17438ce94d4 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:45.931487 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad466ae-bba1-4979-bfd3-6da699c64b04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:45.933404 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad466ae-bba1-4979-bfd3-6da699c64b04 service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:45.944827 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1141/2295] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:40:45 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:40:45.999158 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:46.004519 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e532dcf3-eb2a-419e-a915-b17438ce94d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6b9ba60aaf8497a9fcd941d8308b11d'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:46.014936 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e532dcf3-eb2a-419e-a915-b17438ce94d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6b9ba60aaf8497a9fcd941d8308b11d'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:46.015901 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e532dcf3-eb2a-419e-a915-b17438ce94d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6b9ba60aaf8497a9fcd941d8308b11d'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:46.037964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e532dcf3-eb2a-419e-a915-b17438ce94d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6b9ba60aaf8497a9fcd941d8308b11d'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:46.037964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e532dcf3-eb2a-419e-a915-b17438ce94d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6b9ba60aaf8497a9fcd941d8308b11d'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:46.042776 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58901036-9035-47dc-9455-fe3d2f62e3df None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:46.044477 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58901036-9035-47dc-9455-fe3d2f62e3df None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:46.045023 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58901036-9035-47dc-9455-fe3d2f62e3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:46.055912 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1155/2296] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:40:45 2022] DELETE /identity/v3/users/e6b9ba60aaf8497a9fcd941d8308b11d => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:46.061002 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d884d-935c-4c23-b571-e805a6f80651 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:46.061503 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d884d-935c-4c23-b571-e805a6f80651 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:46.061996 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d884d-935c-4c23-b571-e805a6f80651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:46.088181 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-929d884d-935c-4c23-b571-e805a6f80651 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:46.090144 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-929d884d-935c-4c23-b571-e805a6f80651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:46.097393 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-58901036-9035-47dc-9455-fe3d2f62e3df None None] MFA Rules not processed for user `2949f24248034487ada1e11bc9a9e4d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:46.098706 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-58901036-9035-47dc-9455-fe3d2f62e3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:46.209522 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1142/2297] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:46.209522 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58901036-9035-47dc-9455-fe3d2f62e3df None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:46.210010 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1156/2298] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:40:46.220730 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-58901036-9035-47dc-9455-fe3d2f62e3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:46.223906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d884d-935c-4c23-b571-e805a6f80651 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:46.225531 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-929d884d-935c-4c23-b571-e805a6f80651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:46.235023 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:46.254363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58901036-9035-47dc-9455-fe3d2f62e3df None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:46.262512 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16cac2-fff5-4632-b276-c6cd0bcbca34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:46.263560 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16cac2-fff5-4632-b276-c6cd0bcbca34 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:46.264171 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16cac2-fff5-4632-b276-c6cd0bcbca34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:46.264522 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16cac2-fff5-4632-b276-c6cd0bcbca34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:46.265331 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c16cac2-fff5-4632-b276-c6cd0bcbca34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:46.269053 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c16cac2-fff5-4632-b276-c6cd0bcbca34 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:46.310109 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1143/2299] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:40:46 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:40:46.314903 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16cac2-fff5-4632-b276-c6cd0bcbca34 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:46.325957 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d884d-935c-4c23-b571-e805a6f80651 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:46.398862 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16cac2-fff5-4632-b276-c6cd0bcbca34 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:46.410383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c6c15-d8b2-4da0-a507-dda1ea5d5a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:46.412760 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c6c15-d8b2-4da0-a507-dda1ea5d5a83 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:46.413835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c6c15-d8b2-4da0-a507-dda1ea5d5a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:46.414212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c6c15-d8b2-4da0-a507-dda1ea5d5a83 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:46.415463 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c6c15-d8b2-4da0-a507-dda1ea5d5a83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:46.429575 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c6c15-d8b2-4da0-a507-dda1ea5d5a83 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:46.430040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5768e-c9c4-4f7f-92ee-d0266c163ace service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:46.430040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5768e-c9c4-4f7f-92ee-d0266c163ace service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:46.430040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5768e-c9c4-4f7f-92ee-d0266c163ace service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:46.430040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5768e-c9c4-4f7f-92ee-d0266c163ace service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:46.442589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5768e-c9c4-4f7f-92ee-d0266c163ace service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:46.445578 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5768e-c9c4-4f7f-92ee-d0266c163ace service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:46.466335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1157/2300] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:40:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:40:46.482293 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1144/2301] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:40:46 2022] POST /identity/v3/users => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:40:46.483699 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5768e-c9c4-4f7f-92ee-d0266c163ace service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:46.506464 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5768e-c9c4-4f7f-92ee-d0266c163ace service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:46.523855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c6ec4-0b3b-4fe3-bb3f-c6038fdf8ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:46.525065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c6ec4-0b3b-4fe3-bb3f-c6038fdf8ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:46.525480 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c6ec4-0b3b-4fe3-bb3f-c6038fdf8ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:46.525861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c6ec4-0b3b-4fe3-bb3f-c6038fdf8ed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:46.527459 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c6ec4-0b3b-4fe3-bb3f-c6038fdf8ed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:46.529229 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c6ec4-0b3b-4fe3-bb3f-c6038fdf8ed7 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:46.546768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1158/2302] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:46.551953 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c6c15-d8b2-4da0-a507-dda1ea5d5a83 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:46.569651 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c6c15-d8b2-4da0-a507-dda1ea5d5a83 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:46.583286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77622e07-f6e2-421c-b69b-210c12f1c12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:46.584190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77622e07-f6e2-421c-b69b-210c12f1c12d admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:46.584417 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77622e07-f6e2-421c-b69b-210c12f1c12d admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:46.584986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77622e07-f6e2-421c-b69b-210c12f1c12d admin admin] PATH_INFO: `/v3/projects/c241af3afd124c6d87f0979bc3746e59/users/e6f24df0f2b24487970afa57fc91363c/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:46.602479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77622e07-f6e2-421c-b69b-210c12f1c12d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c241af3afd124c6d87f0979bc3746e59, user_id=e6f24df0f2b24487970afa57fc91363c, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:46.605385 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77622e07-f6e2-421c-b69b-210c12f1c12d admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:46.625869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1145/2303] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:40:46 2022] PUT /identity/v3/projects/c241af3afd124c6d87f0979bc3746e59/users/e6f24df0f2b24487970afa57fc91363c/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:46.630107 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c6ec4-0b3b-4fe3-bb3f-c6038fdf8ed7 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:46.646109 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c6ec4-0b3b-4fe3-bb3f-c6038fdf8ed7 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:46.663449 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:46.781064 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53119c10-0153-4f1d-9337-4d6a097ea1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:46.781951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53119c10-0153-4f1d-9337-4d6a097ea1bb admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:46.782405 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53119c10-0153-4f1d-9337-4d6a097ea1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:46.782823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53119c10-0153-4f1d-9337-4d6a097ea1bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:46.783695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53119c10-0153-4f1d-9337-4d6a097ea1bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:46.786075 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53119c10-0153-4f1d-9337-4d6a097ea1bb admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:46.804610 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1159/2304] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:46.805710 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77622e07-f6e2-421c-b69b-210c12f1c12d admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:46.818040 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77622e07-f6e2-421c-b69b-210c12f1c12d admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:46.824267 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ffee3-cf0c-4320-8415-1b4b1bc454ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:46.825063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ffee3-cf0c-4320-8415-1b4b1bc454ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:46.825421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ffee3-cf0c-4320-8415-1b4b1bc454ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:46.825860 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ffee3-cf0c-4320-8415-1b4b1bc454ec admin admin] PATH_INFO: `/v3/projects/c241af3afd124c6d87f0979bc3746e59/users/e6f24df0f2b24487970afa57fc91363c/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:46.828502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ffee3-cf0c-4320-8415-1b4b1bc454ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=c241af3afd124c6d87f0979bc3746e59, user_id=e6f24df0f2b24487970afa57fc91363c, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:46.831407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ffee3-cf0c-4320-8415-1b4b1bc454ec admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:46.847948 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1146/2305] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:40:46 2022] PUT /identity/v3/projects/c241af3afd124c6d87f0979bc3746e59/users/e6f24df0f2b24487970afa57fc91363c/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:40:46.883274 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53119c10-0153-4f1d-9337-4d6a097ea1bb admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:46.886022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-53119c10-0153-4f1d-9337-4d6a097ea1bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:46.901707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53119c10-0153-4f1d-9337-4d6a097ea1bb admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:46.997583 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945346d-ce6c-4785-b1ce-4b27986342ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:46.998601 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945346d-ce6c-4785-b1ce-4b27986342ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:46.999144 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945346d-ce6c-4785-b1ce-4b27986342ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:46.999618 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945346d-ce6c-4785-b1ce-4b27986342ed admin admin] PATH_INFO: `/v3/projects/b2ce617c0ceb49c3ba0e23d5d315a769` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:47.003185 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945346d-ce6c-4785-b1ce-4b27986342ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2ce617c0ceb49c3ba0e23d5d315a769)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:47.008774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945346d-ce6c-4785-b1ce-4b27986342ed admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:47.085301 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e945346d-ce6c-4785-b1ce-4b27986342ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2ce617c0ceb49c3ba0e23d5d315a769', 'request_id': 'req-e945346d-ce6c-4785-b1ce-4b27986342ed'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:47.095364 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e945346d-ce6c-4785-b1ce-4b27986342ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2ce617c0ceb49c3ba0e23d5d315a769', 'request_id': 'req-e945346d-ce6c-4785-b1ce-4b27986342ed'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:47.103301 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e945346d-ce6c-4785-b1ce-4b27986342ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2ce617c0ceb49c3ba0e23d5d315a769', 'request_id': 'req-e945346d-ce6c-4785-b1ce-4b27986342ed'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:47.130770 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e945346d-ce6c-4785-b1ce-4b27986342ed admin admin] The token cache is being invalidate because project b2ce617c0ceb49c3ba0e23d5d315a769 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:40:47.131069 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e945346d-ce6c-4785-b1ce-4b27986342ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:47.131823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e945346d-ce6c-4785-b1ce-4b27986342ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:47.137339 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1160/2306] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:40:46 2022] DELETE /identity/v3/projects/b2ce617c0ceb49c3ba0e23d5d315a769 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:47.138457 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ffee3-cf0c-4320-8415-1b4b1bc454ec admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:47.140844 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-835ffee3-cf0c-4320-8415-1b4b1bc454ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:47.160052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ffee3-cf0c-4320-8415-1b4b1bc454ec admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:47.384251 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb9524-d80c-4f3c-a605-a3cc9c1f95b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:47.385257 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb9524-d80c-4f3c-a605-a3cc9c1f95b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:47.386485 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb9524-d80c-4f3c-a605-a3cc9c1f95b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:47.387253 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb9524-d80c-4f3c-a605-a3cc9c1f95b3 admin admin] PATH_INFO: `/v3/users/38074f0063754bd7bfcca31d8170a941` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:47.392547 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cb9524-d80c-4f3c-a605-a3cc9c1f95b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38074f0063754bd7bfcca31d8170a941)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:47.394595 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cb9524-d80c-4f3c-a605-a3cc9c1f95b3 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:47.584668 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:INSERT=6,keystone:DELETE=5 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:47.608361 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-14cb9524-d80c-4f3c-a605-a3cc9c1f95b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38074f0063754bd7bfcca31d8170a941'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:47.609434 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-14cb9524-d80c-4f3c-a605-a3cc9c1f95b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38074f0063754bd7bfcca31d8170a941'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:47.628397 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-14cb9524-d80c-4f3c-a605-a3cc9c1f95b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38074f0063754bd7bfcca31d8170a941'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:47.641854 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-14cb9524-d80c-4f3c-a605-a3cc9c1f95b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38074f0063754bd7bfcca31d8170a941'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:47.642572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-14cb9524-d80c-4f3c-a605-a3cc9c1f95b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38074f0063754bd7bfcca31d8170a941'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:47.662646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1147/2307] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:40:47 2022] DELETE /identity/v3/users/38074f0063754bd7bfcca31d8170a941 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:48.746635 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945346d-ce6c-4785-b1ce-4b27986342ed admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:48.771918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945346d-ce6c-4785-b1ce-4b27986342ed admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:48.866333 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:48.910257 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299e5b5-8d82-4207-9d30-fd8f6a40d868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:48.910930 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299e5b5-8d82-4207-9d30-fd8f6a40d868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:48.911207 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299e5b5-8d82-4207-9d30-fd8f6a40d868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:48.911483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299e5b5-8d82-4207-9d30-fd8f6a40d868 admin admin] PATH_INFO: `/v3/projects/6a92c9c6efd24b9ea5ca3ca69320204a` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:48.913882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299e5b5-8d82-4207-9d30-fd8f6a40d868 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a92c9c6efd24b9ea5ca3ca69320204a)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:48.918287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299e5b5-8d82-4207-9d30-fd8f6a40d868 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:49.033271 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4299e5b5-8d82-4207-9d30-fd8f6a40d868 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a92c9c6efd24b9ea5ca3ca69320204a', 'request_id': 'req-4299e5b5-8d82-4207-9d30-fd8f6a40d868'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:49.045849 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4299e5b5-8d82-4207-9d30-fd8f6a40d868 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a92c9c6efd24b9ea5ca3ca69320204a', 'request_id': 'req-4299e5b5-8d82-4207-9d30-fd8f6a40d868'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:49.054760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4299e5b5-8d82-4207-9d30-fd8f6a40d868 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a92c9c6efd24b9ea5ca3ca69320204a', 'request_id': 'req-4299e5b5-8d82-4207-9d30-fd8f6a40d868'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:49.083787 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4299e5b5-8d82-4207-9d30-fd8f6a40d868 admin admin] The token cache is being invalidate because project 6a92c9c6efd24b9ea5ca3ca69320204a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:40:49.084077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4299e5b5-8d82-4207-9d30-fd8f6a40d868 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:49.084791 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4299e5b5-8d82-4207-9d30-fd8f6a40d868 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:49.086667 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1161/2308] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:40:48 2022] DELETE /identity/v3/projects/6a92c9c6efd24b9ea5ca3ca69320204a => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:49.093421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2ac14-b62c-4452-b013-ef44f536cd76 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:49.094344 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2ac14-b62c-4452-b013-ef44f536cd76 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:49.094762 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2ac14-b62c-4452-b013-ef44f536cd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:49.131406 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-3ca2ac14-b62c-4452-b013-ef44f536cd76 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:49.132692 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-3ca2ac14-b62c-4452-b013-ef44f536cd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:49.221828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1148/2309] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:49.230048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21639917-c285-4609-aa27-8b9f0a14a7e0 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:49.230179 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21639917-c285-4609-aa27-8b9f0a14a7e0 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:49.230403 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21639917-c285-4609-aa27-8b9f0a14a7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:49.257719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-21639917-c285-4609-aa27-8b9f0a14a7e0 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:49.259753 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-21639917-c285-4609-aa27-8b9f0a14a7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:49.283275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1162/2310] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:49.288015 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca2ac14-b62c-4452-b013-ef44f536cd76 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:49.302555 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca2ac14-b62c-4452-b013-ef44f536cd76 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:49.313746 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6418a9-ca2f-4cb5-9910-df7a64733194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:49.314491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6418a9-ca2f-4cb5-9910-df7a64733194 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:49.315253 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6418a9-ca2f-4cb5-9910-df7a64733194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:49.315253 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6418a9-ca2f-4cb5-9910-df7a64733194 admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:49.315974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6418a9-ca2f-4cb5-9910-df7a64733194 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:40:49.316342 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6418a9-ca2f-4cb5-9910-df7a64733194 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:49.318382 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6418a9-ca2f-4cb5-9910-df7a64733194 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:49.341826 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1149/2311] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:40:49.349009 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21639917-c285-4609-aa27-8b9f0a14a7e0 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:49.355330 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21639917-c285-4609-aa27-8b9f0a14a7e0 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:49.368721 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a29ae9-fe3b-4847-908e-c6e018c67d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:49.369916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a29ae9-fe3b-4847-908e-c6e018c67d1c admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:49.370309 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a29ae9-fe3b-4847-908e-c6e018c67d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:49.370974 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a29ae9-fe3b-4847-908e-c6e018c67d1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:49.371469 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a29ae9-fe3b-4847-908e-c6e018c67d1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:49.373990 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a29ae9-fe3b-4847-908e-c6e018c67d1c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:49.445889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6418a9-ca2f-4cb5-9910-df7a64733194 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:49.455096 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1163/2312] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:40:49 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:40:49.460243 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6418a9-ca2f-4cb5-9910-df7a64733194 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:49.796626 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2631bb-cecb-4cc4-b7ed-cb1cc72508f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:49.799196 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2631bb-cecb-4cc4-b7ed-cb1cc72508f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:49.799196 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2631bb-cecb-4cc4-b7ed-cb1cc72508f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:49.799442 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2631bb-cecb-4cc4-b7ed-cb1cc72508f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:49.802526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2631bb-cecb-4cc4-b7ed-cb1cc72508f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:49.805868 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2631bb-cecb-4cc4-b7ed-cb1cc72508f7 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:49.839399 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1150/2313] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:40:49 2022] POST /identity/v3/users => generated 289 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:40:49.843262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a29ae9-fe3b-4847-908e-c6e018c67d1c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:49.856016 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a29ae9-fe3b-4847-908e-c6e018c67d1c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:49.862390 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e704bec-eb12-464a-8dc8-50c0c1fcce95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:49.863034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e704bec-eb12-464a-8dc8-50c0c1fcce95 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:49.863313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e704bec-eb12-464a-8dc8-50c0c1fcce95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:49.864818 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e704bec-eb12-464a-8dc8-50c0c1fcce95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:49.864818 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e704bec-eb12-464a-8dc8-50c0c1fcce95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:49.866660 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e704bec-eb12-464a-8dc8-50c0c1fcce95 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:49.880072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1164/2314] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:49.885278 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2631bb-cecb-4cc4-b7ed-cb1cc72508f7 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:49.898617 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2631bb-cecb-4cc4-b7ed-cb1cc72508f7 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:49.905349 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e2ae35-3a4b-4115-944f-2daabbf88d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:49.905827 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2ae35-3a4b-4115-944f-2daabbf88d6a admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:49.906112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2ae35-3a4b-4115-944f-2daabbf88d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:49.906385 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2ae35-3a4b-4115-944f-2daabbf88d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:49.907212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e2ae35-3a4b-4115-944f-2daabbf88d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:49.908844 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e2ae35-3a4b-4115-944f-2daabbf88d6a admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:49.925746 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1151/2315] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:40:49.929475 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e704bec-eb12-464a-8dc8-50c0c1fcce95 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:49.953687 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e704bec-eb12-464a-8dc8-50c0c1fcce95 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:49.964681 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf951b2-bd79-493c-9403-082a0f89977d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:49.966093 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf951b2-bd79-493c-9403-082a0f89977d admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:49.966614 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf951b2-bd79-493c-9403-082a0f89977d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:49.967245 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf951b2-bd79-493c-9403-082a0f89977d admin admin] PATH_INFO: `/v3/projects/daa9c4eb590c4b28b0027e8645d33a68/users/45f0ba4c03e74096bfd62adbf1db9a16/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:50.004515 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf951b2-bd79-493c-9403-082a0f89977d admin admin] RBAC: Authorizing `identity:create_grant(project_id=daa9c4eb590c4b28b0027e8645d33a68, user_id=45f0ba4c03e74096bfd62adbf1db9a16, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:50.010051 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf951b2-bd79-493c-9403-082a0f89977d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:50.034428 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1165/2316] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:40:49 2022] PUT /identity/v3/projects/daa9c4eb590c4b28b0027e8645d33a68/users/45f0ba4c03e74096bfd62adbf1db9a16/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:50.055276 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f952c8a-f05e-4ab5-9a30-6b6a65f337a2 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:50.055276 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f952c8a-f05e-4ab5-9a30-6b6a65f337a2 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:50.056849 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f952c8a-f05e-4ab5-9a30-6b6a65f337a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:50.110168 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-3f952c8a-f05e-4ab5-9a30-6b6a65f337a2 None None] MFA Rules not processed for user `45f0ba4c03e74096bfd62adbf1db9a16`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:50.112050 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-3f952c8a-f05e-4ab5-9a30-6b6a65f337a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:50.142797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:50.272524 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1152/2317] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:40:50.289084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf951b2-bd79-493c-9403-082a0f89977d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:50.292619 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ecf951b2-bd79-493c-9403-082a0f89977d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:50.452803 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:50.465381 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf951b2-bd79-493c-9403-082a0f89977d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:50.471013 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82137c3-e5fa-4317-9812-657f6cb7a63a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b77a955866c54a43a7aaa271fefdec57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:50.471936 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82137c3-e5fa-4317-9812-657f6cb7a63a service glance] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:50.472301 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82137c3-e5fa-4317-9812-657f6cb7a63a service glance] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:50.472981 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82137c3-e5fa-4317-9812-657f6cb7a63a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:50.488254 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82137c3-e5fa-4317-9812-657f6cb7a63a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:50.490048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82137c3-e5fa-4317-9812-657f6cb7a63a service glance] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:50.502140 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1166/2318] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:40:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:40:52.091471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f952c8a-f05e-4ab5-9a30-6b6a65f337a2 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:52.094136 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-3f952c8a-f05e-4ab5-9a30-6b6a65f337a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:52.190509 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f952c8a-f05e-4ab5-9a30-6b6a65f337a2 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:52.196708 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16056f30-6817-4888-a4ee-df71e32a6b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:52.197844 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16056f30-6817-4888-a4ee-df71e32a6b90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:52.198255 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16056f30-6817-4888-a4ee-df71e32a6b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:52.198837 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16056f30-6817-4888-a4ee-df71e32a6b90 admin admin] PATH_INFO: `/v3/users/52ec3d2aa0e347e3bf7dcfa68743b059` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:52.204816 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16056f30-6817-4888-a4ee-df71e32a6b90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52ec3d2aa0e347e3bf7dcfa68743b059)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:52.204816 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16056f30-6817-4888-a4ee-df71e32a6b90 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:52.309858 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-16056f30-6817-4888-a4ee-df71e32a6b90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52ec3d2aa0e347e3bf7dcfa68743b059'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:52.310618 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-16056f30-6817-4888-a4ee-df71e32a6b90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52ec3d2aa0e347e3bf7dcfa68743b059'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:52.342016 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-16056f30-6817-4888-a4ee-df71e32a6b90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52ec3d2aa0e347e3bf7dcfa68743b059'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:52.355920 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-16056f30-6817-4888-a4ee-df71e32a6b90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52ec3d2aa0e347e3bf7dcfa68743b059'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:52.357882 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-16056f30-6817-4888-a4ee-df71e32a6b90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52ec3d2aa0e347e3bf7dcfa68743b059'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:52.386192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1153/2319] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:40:52 2022] DELETE /identity/v3/users/52ec3d2aa0e347e3bf7dcfa68743b059 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:52.386297 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d644abb-0298-48d3-84fa-6de266a7f917 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:52.386297 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d644abb-0298-48d3-84fa-6de266a7f917 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:52.386297 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d644abb-0298-48d3-84fa-6de266a7f917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:52.412704 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-1d644abb-0298-48d3-84fa-6de266a7f917 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:52.414858 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1d644abb-0298-48d3-84fa-6de266a7f917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:52.589494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1167/2320] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:52.599131 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16056f30-6817-4888-a4ee-df71e32a6b90 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:52.602928 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-16056f30-6817-4888-a4ee-df71e32a6b90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:52.723811 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:52.785764 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16056f30-6817-4888-a4ee-df71e32a6b90 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:52.790287 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf427f-4e71-4cc1-b643-d85ee1c548a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:52.791256 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf427f-4e71-4cc1-b643-d85ee1c548a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:52.791679 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf427f-4e71-4cc1-b643-d85ee1c548a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:52.791962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf427f-4e71-4cc1-b643-d85ee1c548a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:52.816079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf427f-4e71-4cc1-b643-d85ee1c548a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:52.817962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf427f-4e71-4cc1-b643-d85ee1c548a3 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:52.847714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1154/2321] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:40:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:40:53.510717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d644abb-0298-48d3-84fa-6de266a7f917 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:53.522294 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1d644abb-0298-48d3-84fa-6de266a7f917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:53.536088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d644abb-0298-48d3-84fa-6de266a7f917 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:53.543472 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dde123-6a4f-4b48-8a8c-0aed7750b131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:53.544779 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dde123-6a4f-4b48-8a8c-0aed7750b131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:53.545188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dde123-6a4f-4b48-8a8c-0aed7750b131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:53.545726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dde123-6a4f-4b48-8a8c-0aed7750b131 admin admin] PATH_INFO: `/v3/projects/9606e6021ff44458900026939e58fdb4` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:53.547291 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dde123-6a4f-4b48-8a8c-0aed7750b131 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9606e6021ff44458900026939e58fdb4)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:53.550224 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dde123-6a4f-4b48-8a8c-0aed7750b131 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:53.646446 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b1dde123-6a4f-4b48-8a8c-0aed7750b131 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9606e6021ff44458900026939e58fdb4', 'request_id': 'req-b1dde123-6a4f-4b48-8a8c-0aed7750b131'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:53.658277 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b1dde123-6a4f-4b48-8a8c-0aed7750b131 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9606e6021ff44458900026939e58fdb4', 'request_id': 'req-b1dde123-6a4f-4b48-8a8c-0aed7750b131'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:53.668914 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b1dde123-6a4f-4b48-8a8c-0aed7750b131 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9606e6021ff44458900026939e58fdb4', 'request_id': 'req-b1dde123-6a4f-4b48-8a8c-0aed7750b131'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:53.691088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b1dde123-6a4f-4b48-8a8c-0aed7750b131 admin admin] The token cache is being invalidate because project 9606e6021ff44458900026939e58fdb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:40:53.693900 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b1dde123-6a4f-4b48-8a8c-0aed7750b131 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:53.694841 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b1dde123-6a4f-4b48-8a8c-0aed7750b131 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:53.700222 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1168/2322] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:40:53 2022] DELETE /identity/v3/projects/9606e6021ff44458900026939e58fdb4 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:53.702129 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf427f-4e71-4cc1-b643-d85ee1c548a3 service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:53.724126 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf427f-4e71-4cc1-b643-d85ee1c548a3 service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:53.802556 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb6521-4b07-4a9d-bb11-9c75fbbbf00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:53.803148 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb6521-4b07-4a9d-bb11-9c75fbbbf00b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:53.803431 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb6521-4b07-4a9d-bb11-9c75fbbbf00b admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:53.803709 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb6521-4b07-4a9d-bb11-9c75fbbbf00b admin admin] PATH_INFO: `/v3/users/cdb2b6e419fc472abbb415b6b5917670` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:53.804941 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb6521-4b07-4a9d-bb11-9c75fbbbf00b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdb2b6e419fc472abbb415b6b5917670)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:53.807656 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb6521-4b07-4a9d-bb11-9c75fbbbf00b admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:53.881988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-cdcb6521-4b07-4a9d-bb11-9c75fbbbf00b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdb2b6e419fc472abbb415b6b5917670'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:53.882854 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-cdcb6521-4b07-4a9d-bb11-9c75fbbbf00b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdb2b6e419fc472abbb415b6b5917670'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:53.908375 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-cdcb6521-4b07-4a9d-bb11-9c75fbbbf00b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdb2b6e419fc472abbb415b6b5917670'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:53.918907 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-cdcb6521-4b07-4a9d-bb11-9c75fbbbf00b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdb2b6e419fc472abbb415b6b5917670'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:53.919556 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-cdcb6521-4b07-4a9d-bb11-9c75fbbbf00b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdb2b6e419fc472abbb415b6b5917670'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:53.946423 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31db507-30f7-47ba-bd98-30bcbf562d2c None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:53.946836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31db507-30f7-47ba-bd98-30bcbf562d2c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:53.947116 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31db507-30f7-47ba-bd98-30bcbf562d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:53.948699 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1155/2323] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:40:53 2022] DELETE /identity/v3/users/cdb2b6e419fc472abbb415b6b5917670 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:53.990164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-c31db507-30f7-47ba-bd98-30bcbf562d2c None None] MFA Rules not processed for user `e6f24df0f2b24487970afa57fc91363c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:53.991400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c31db507-30f7-47ba-bd98-30bcbf562d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:54.081747 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:54.096197 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1169/2324] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:40:54.113855 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb6521-4b07-4a9d-bb11-9c75fbbbf00b admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:54.115502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-cdcb6521-4b07-4a9d-bb11-9c75fbbbf00b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:54.256472 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:54.270210 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb6521-4b07-4a9d-bb11-9c75fbbbf00b admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:54.276872 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964e0d2-97ac-4b08-a6d8-8ad6c93f47db service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b77a955866c54a43a7aaa271fefdec57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:54.278486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964e0d2-97ac-4b08-a6d8-8ad6c93f47db service glance] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:54.279683 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964e0d2-97ac-4b08-a6d8-8ad6c93f47db service glance] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:54.280073 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964e0d2-97ac-4b08-a6d8-8ad6c93f47db service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:54.295500 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-e964e0d2-97ac-4b08-a6d8-8ad6c93f47db service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:54.396943 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e964e0d2-97ac-4b08-a6d8-8ad6c93f47db service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:54.398674 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e964e0d2-97ac-4b08-a6d8-8ad6c93f47db service glance] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:54.454691 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1156/2325] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:40:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:40:54.517432 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31db507-30f7-47ba-bd98-30bcbf562d2c None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:54.533872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c31db507-30f7-47ba-bd98-30bcbf562d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:54.646466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31db507-30f7-47ba-bd98-30bcbf562d2c None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:54.653271 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd7c3c-86ee-41a2-b810-dbe74dadf485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:54.654344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd7c3c-86ee-41a2-b810-dbe74dadf485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:54.656217 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd7c3c-86ee-41a2-b810-dbe74dadf485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:54.656675 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd7c3c-86ee-41a2-b810-dbe74dadf485 admin admin] PATH_INFO: `/v3/projects/146da10b6b6346f683aef5cc71f84a63` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:54.658710 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd7c3c-86ee-41a2-b810-dbe74dadf485 admin admin] RBAC: Authorizing `identity:delete_project(project_id=146da10b6b6346f683aef5cc71f84a63)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:54.662926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd7c3c-86ee-41a2-b810-dbe74dadf485 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:54.669929 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921cc64-3557-4150-a405-30daf38f524d None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:54.670239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921cc64-3557-4150-a405-30daf38f524d None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:54.670719 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921cc64-3557-4150-a405-30daf38f524d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:54.702273 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-e921cc64-3557-4150-a405-30daf38f524d None None] MFA Rules not processed for user `3a721708c7e443e799349add27de8f84`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:54.707445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-e921cc64-3557-4150-a405-30daf38f524d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:54.763274 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e0fd7c3c-86ee-41a2-b810-dbe74dadf485 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '146da10b6b6346f683aef5cc71f84a63', 'request_id': 'req-e0fd7c3c-86ee-41a2-b810-dbe74dadf485'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:54.774204 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e0fd7c3c-86ee-41a2-b810-dbe74dadf485 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '146da10b6b6346f683aef5cc71f84a63', 'request_id': 'req-e0fd7c3c-86ee-41a2-b810-dbe74dadf485'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:54.784006 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e0fd7c3c-86ee-41a2-b810-dbe74dadf485 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '146da10b6b6346f683aef5cc71f84a63', 'request_id': 'req-e0fd7c3c-86ee-41a2-b810-dbe74dadf485'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:54.798983 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e0fd7c3c-86ee-41a2-b810-dbe74dadf485 admin admin] The token cache is being invalidate because project 146da10b6b6346f683aef5cc71f84a63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:40:54.799283 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e0fd7c3c-86ee-41a2-b810-dbe74dadf485 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:54.800171 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e0fd7c3c-86ee-41a2-b810-dbe74dadf485 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:40:54.802237 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1170/2326] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:40:54 2022] DELETE /identity/v3/projects/146da10b6b6346f683aef5cc71f84a63 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:40:54.865204 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1157/2327] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Aug 4 04:40:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:40:54.879223 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd7c3c-86ee-41a2-b810-dbe74dadf485 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:40:54.881486 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e0fd7c3c-86ee-41a2-b810-dbe74dadf485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:54.980805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd7c3c-86ee-41a2-b810-dbe74dadf485 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:40:54.986911 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a19c8-03d7-4a77-9d0a-7c7e723639c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1e539006166434790823b839bd745d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:40:54.987834 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a19c8-03d7-4a77-9d0a-7c7e723639c0 service swift] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:54.988122 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a19c8-03d7-4a77-9d0a-7c7e723639c0 service swift] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:54.989829 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a19c8-03d7-4a77-9d0a-7c7e723639c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:55.048125 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:40:55.100032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a19c8-03d7-4a77-9d0a-7c7e723639c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:40:55.102421 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a19c8-03d7-4a77-9d0a-7c7e723639c0 service swift] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:40:55.114438 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1171/2328] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Aug 4 04:40:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 04:40:59.308764 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d2260-8109-4803-aade-1dcd68eb3465 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:40:59.309617 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d2260-8109-4803-aade-1dcd68eb3465 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:40:59.310004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d2260-8109-4803-aade-1dcd68eb3465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:40:59.351138 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-807d2260-8109-4803-aade-1dcd68eb3465 None None] MFA Rules not processed for user `49a7ce9a27734dc0a187d1b217206785`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:40:59.355383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-807d2260-8109-4803-aade-1dcd68eb3465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:40:59.448465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1158/2329] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:40:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:41:05.108347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=15 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:41:06.530806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571814f6-774a-4d85-ab25-0e64c5a71fbb None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:06.531975 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571814f6-774a-4d85-ab25-0e64c5a71fbb None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:06.532474 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571814f6-774a-4d85-ab25-0e64c5a71fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:06.588622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-571814f6-774a-4d85-ab25-0e64c5a71fbb None None] MFA Rules not processed for user `3a721708c7e443e799349add27de8f84`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:41:06.590386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-571814f6-774a-4d85-ab25-0e64c5a71fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:06.600824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1172/2330] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Aug 4 04:41:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:41:06.621039 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d2260-8109-4803-aade-1dcd68eb3465 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:06.639917 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d2260-8109-4803-aade-1dcd68eb3465 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:06.647296 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393f5e0-5369-4e6d-92ff-3aa81fd076d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1e539006166434790823b839bd745d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:06.647955 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393f5e0-5369-4e6d-92ff-3aa81fd076d3 service swift] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:06.648766 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393f5e0-5369-4e6d-92ff-3aa81fd076d3 service swift] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:06.650100 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393f5e0-5369-4e6d-92ff-3aa81fd076d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:06.660207 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0393f5e0-5369-4e6d-92ff-3aa81fd076d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:06.661857 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0393f5e0-5369-4e6d-92ff-3aa81fd076d3 service swift] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:06.673585 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1159/2331] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Aug 4 04:41:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 04:41:07.609757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340eb9f9-10fc-4474-b5c3-7ede3a53af63 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:07.610229 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340eb9f9-10fc-4474-b5c3-7ede3a53af63 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:07.610317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340eb9f9-10fc-4474-b5c3-7ede3a53af63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:07.650802 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-340eb9f9-10fc-4474-b5c3-7ede3a53af63 None None] MFA Rules not processed for user `2949f24248034487ada1e11bc9a9e4d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:41:07.651980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-340eb9f9-10fc-4474-b5c3-7ede3a53af63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:07.812145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1173/2332] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:41:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:41:12.262060 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393f5e0-5369-4e6d-92ff-3aa81fd076d3 service swift] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:12.268207 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-0393f5e0-5369-4e6d-92ff-3aa81fd076d3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:12.370022 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:41:12.401378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393f5e0-5369-4e6d-92ff-3aa81fd076d3 service swift] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:12.408896 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d590b99-c010-4796-a790-b7916b9aa15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:12.409747 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d590b99-c010-4796-a790-b7916b9aa15c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:12.410393 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d590b99-c010-4796-a790-b7916b9aa15c admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:12.411330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d590b99-c010-4796-a790-b7916b9aa15c admin admin] PATH_INFO: `/v3/users/45f0ba4c03e74096bfd62adbf1db9a16` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:12.412247 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d590b99-c010-4796-a790-b7916b9aa15c admin admin] RBAC: Authorizing `identity:delete_user(user_id=45f0ba4c03e74096bfd62adbf1db9a16)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:12.414378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d590b99-c010-4796-a790-b7916b9aa15c admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:12.496813 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9d590b99-c010-4796-a790-b7916b9aa15c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45f0ba4c03e74096bfd62adbf1db9a16'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:12.497771 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9d590b99-c010-4796-a790-b7916b9aa15c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45f0ba4c03e74096bfd62adbf1db9a16'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:12.513839 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9d590b99-c010-4796-a790-b7916b9aa15c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45f0ba4c03e74096bfd62adbf1db9a16'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:12.522638 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9d590b99-c010-4796-a790-b7916b9aa15c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45f0ba4c03e74096bfd62adbf1db9a16'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:12.523558 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9d590b99-c010-4796-a790-b7916b9aa15c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45f0ba4c03e74096bfd62adbf1db9a16'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:12.539650 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1160/2333] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:41:12 2022] DELETE /identity/v3/users/45f0ba4c03e74096bfd62adbf1db9a16 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:41:12.544541 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cde43-4de3-4a80-88e7-4a2969e97fe2 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:12.545093 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cde43-4de3-4a80-88e7-4a2969e97fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:12.545093 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cde43-4de3-4a80-88e7-4a2969e97fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:12.566843 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b69cde43-4de3-4a80-88e7-4a2969e97fe2 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:41:12.568103 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b69cde43-4de3-4a80-88e7-4a2969e97fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:12.660507 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1174/2334] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:41:12.673853 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d590b99-c010-4796-a790-b7916b9aa15c admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:12.676190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-9d590b99-c010-4796-a790-b7916b9aa15c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:12.818009 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d590b99-c010-4796-a790-b7916b9aa15c admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:12.844021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d221b27-c3c4-47b1-8dd6-6651506c65a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:12.844670 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d221b27-c3c4-47b1-8dd6-6651506c65a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:12.844954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d221b27-c3c4-47b1-8dd6-6651506c65a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:12.847235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d221b27-c3c4-47b1-8dd6-6651506c65a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:12.885394 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d221b27-c3c4-47b1-8dd6-6651506c65a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:12.887234 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d221b27-c3c4-47b1-8dd6-6651506c65a7 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:12.907935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1161/2335] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:41:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:41:13.084958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48b8be-e32c-4390-8503-877e99158155 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:13.085318 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48b8be-e32c-4390-8503-877e99158155 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:13.086233 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48b8be-e32c-4390-8503-877e99158155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:13.135464 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-2e48b8be-e32c-4390-8503-877e99158155 None None] MFA Rules not processed for user `3a721708c7e443e799349add27de8f84`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:41:13.138159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2e48b8be-e32c-4390-8503-877e99158155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:13.276501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1175/2336] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Aug 4 04:41:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:41:13.308491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d221b27-c3c4-47b1-8dd6-6651506c65a7 service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:13.311498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-7d221b27-c3c4-47b1-8dd6-6651506c65a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:13.554826 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:41:13.566964 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d221b27-c3c4-47b1-8dd6-6651506c65a7 service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:13.575761 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295cf1d8-5cc9-43e5-b4d7-d737b3c688be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1e539006166434790823b839bd745d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:13.577350 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295cf1d8-5cc9-43e5-b4d7-d737b3c688be service swift] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:13.577814 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295cf1d8-5cc9-43e5-b4d7-d737b3c688be service swift] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:13.578105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295cf1d8-5cc9-43e5-b4d7-d737b3c688be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:13.598146 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295cf1d8-5cc9-43e5-b4d7-d737b3c688be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:13.599858 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295cf1d8-5cc9-43e5-b4d7-d737b3c688be service swift] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:13.613166 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1162/2337] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Aug 4 04:41:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 04:41:13.646459 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48b8be-e32c-4390-8503-877e99158155 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:13.648575 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2e48b8be-e32c-4390-8503-877e99158155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:13.679235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48b8be-e32c-4390-8503-877e99158155 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:13.697098 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a23045e-6c74-4ce8-9d91-b933ed58dc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:13.697983 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23045e-6c74-4ce8-9d91-b933ed58dc96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:13.698373 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23045e-6c74-4ce8-9d91-b933ed58dc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:13.698765 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23045e-6c74-4ce8-9d91-b933ed58dc96 admin admin] PATH_INFO: `/v3/projects/daa9c4eb590c4b28b0027e8645d33a68` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:13.701767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a23045e-6c74-4ce8-9d91-b933ed58dc96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daa9c4eb590c4b28b0027e8645d33a68)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:13.704458 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a23045e-6c74-4ce8-9d91-b933ed58dc96 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:13.813707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:41:13.832010 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-3a23045e-6c74-4ce8-9d91-b933ed58dc96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daa9c4eb590c4b28b0027e8645d33a68', 'request_id': 'req-3a23045e-6c74-4ce8-9d91-b933ed58dc96'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:13.878357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-3a23045e-6c74-4ce8-9d91-b933ed58dc96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daa9c4eb590c4b28b0027e8645d33a68', 'request_id': 'req-3a23045e-6c74-4ce8-9d91-b933ed58dc96'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:13.890621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-3a23045e-6c74-4ce8-9d91-b933ed58dc96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daa9c4eb590c4b28b0027e8645d33a68', 'request_id': 'req-3a23045e-6c74-4ce8-9d91-b933ed58dc96'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:13.918850 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-3a23045e-6c74-4ce8-9d91-b933ed58dc96 admin admin] The token cache is being invalidate because project daa9c4eb590c4b28b0027e8645d33a68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:41:13.919599 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-3a23045e-6c74-4ce8-9d91-b933ed58dc96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:13.921959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-3a23045e-6c74-4ce8-9d91-b933ed58dc96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:13.932463 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5d2f5-361a-41f8-a0bf-ee59df20ae1d None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:13.933672 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5d2f5-361a-41f8-a0bf-ee59df20ae1d None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:13.934170 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5d2f5-361a-41f8-a0bf-ee59df20ae1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:13.937142 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1176/2338] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:41:13 2022] DELETE /identity/v3/projects/daa9c4eb590c4b28b0027e8645d33a68 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:41:14.060508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-9ab5d2f5-361a-41f8-a0bf-ee59df20ae1d None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:41:14.060508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-9ab5d2f5-361a-41f8-a0bf-ee59df20ae1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:14.300360 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1163/2339] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:41:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:41:14.311213 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147c26e-28f7-47e8-a1c2-47b3b428f439 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:14.311528 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147c26e-28f7-47e8-a1c2-47b3b428f439 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:14.311790 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147c26e-28f7-47e8-a1c2-47b3b428f439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:14.360484 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-2147c26e-28f7-47e8-a1c2-47b3b428f439 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:41:14.361869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2147c26e-28f7-47e8-a1c2-47b3b428f439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:14.386524 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1177/2340] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:41:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:41:14.391340 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5d2f5-361a-41f8-a0bf-ee59df20ae1d None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:14.405415 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5d2f5-361a-41f8-a0bf-ee59df20ae1d None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:14.415463 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542432b-4d53-4588-a4e0-a8d5c06da7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:14.416894 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542432b-4d53-4588-a4e0-a8d5c06da7ae admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:14.417703 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542432b-4d53-4588-a4e0-a8d5c06da7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:14.418147 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542432b-4d53-4588-a4e0-a8d5c06da7ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:14.419608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542432b-4d53-4588-a4e0-a8d5c06da7ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:41:14.420118 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542432b-4d53-4588-a4e0-a8d5c06da7ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:14.422205 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542432b-4d53-4588-a4e0-a8d5c06da7ae admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:14.445930 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147c26e-28f7-47e8-a1c2-47b3b428f439 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:14.451706 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1164/2341] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:41:14.460392 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147c26e-28f7-47e8-a1c2-47b3b428f439 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:14.467700 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160ad4b-0388-4cdc-8aa4-63ba349e033f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:14.468583 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160ad4b-0388-4cdc-8aa4-63ba349e033f admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:14.469279 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160ad4b-0388-4cdc-8aa4-63ba349e033f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:14.469902 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160ad4b-0388-4cdc-8aa4-63ba349e033f admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:14.470933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d160ad4b-0388-4cdc-8aa4-63ba349e033f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:14.473657 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d160ad4b-0388-4cdc-8aa4-63ba349e033f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:14.530699 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542432b-4d53-4588-a4e0-a8d5c06da7ae admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:14.531781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1178/2342] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:41:14 2022] POST /identity/v3/projects => generated 377 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:41:14.542914 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542432b-4d53-4588-a4e0-a8d5c06da7ae admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:14.660640 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e847c9-6356-49eb-957a-f414756b4640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:14.661295 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e847c9-6356-49eb-957a-f414756b4640 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:14.661927 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e847c9-6356-49eb-957a-f414756b4640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:14.662362 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e847c9-6356-49eb-957a-f414756b4640 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:14.663064 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e847c9-6356-49eb-957a-f414756b4640 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:14.665135 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e847c9-6356-49eb-957a-f414756b4640 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:14.792191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1165/2343] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:41:14 2022] POST /identity/v3/users => generated 289 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:41:14.797159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160ad4b-0388-4cdc-8aa4-63ba349e033f admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:14.814486 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160ad4b-0388-4cdc-8aa4-63ba349e033f admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:14.837820 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b7cd6-49fa-4f06-b471-607b12ae023f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:14.838765 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b7cd6-49fa-4f06-b471-607b12ae023f admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:14.839222 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b7cd6-49fa-4f06-b471-607b12ae023f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:14.839657 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b7cd6-49fa-4f06-b471-607b12ae023f admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:14.840540 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b7cd6-49fa-4f06-b471-607b12ae023f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:14.842376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b7cd6-49fa-4f06-b471-607b12ae023f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:14.870006 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1179/2344] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:41:14.871365 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e847c9-6356-49eb-957a-f414756b4640 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:14.905063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e847c9-6356-49eb-957a-f414756b4640 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:14.925204 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc64386-fae1-4b7a-a4cf-9442b2c465b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:14.926586 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc64386-fae1-4b7a-a4cf-9442b2c465b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:14.927084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc64386-fae1-4b7a-a4cf-9442b2c465b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:14.927576 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc64386-fae1-4b7a-a4cf-9442b2c465b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:14.928603 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc64386-fae1-4b7a-a4cf-9442b2c465b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:14.930647 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc64386-fae1-4b7a-a4cf-9442b2c465b6 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:14.949526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1166/2345] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 04:41:14.951452 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b7cd6-49fa-4f06-b471-607b12ae023f admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:14.970131 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b7cd6-49fa-4f06-b471-607b12ae023f admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:14.984986 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92337dc5-7ee3-4266-9282-79ac5941e828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:14.985726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92337dc5-7ee3-4266-9282-79ac5941e828 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:14.987473 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92337dc5-7ee3-4266-9282-79ac5941e828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:14.987948 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92337dc5-7ee3-4266-9282-79ac5941e828 admin admin] PATH_INFO: `/v3/projects/f775bbf7216a4717bc9163cedb31b7cf/users/f31954ad642e4ac0a3371f1910510b7a/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:15.045652 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92337dc5-7ee3-4266-9282-79ac5941e828 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f775bbf7216a4717bc9163cedb31b7cf, user_id=f31954ad642e4ac0a3371f1910510b7a, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:15.048572 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92337dc5-7ee3-4266-9282-79ac5941e828 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:15.077505 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1180/2346] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:41:14 2022] PUT /identity/v3/projects/f775bbf7216a4717bc9163cedb31b7cf/users/f31954ad642e4ac0a3371f1910510b7a/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:41:15.088037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c03ba-93b0-40ce-a4a1-93efc75ee50d None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:15.088746 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c03ba-93b0-40ce-a4a1-93efc75ee50d None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:15.089208 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c03ba-93b0-40ce-a4a1-93efc75ee50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:15.173464 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-f76c03ba-93b0-40ce-a4a1-93efc75ee50d None None] MFA Rules not processed for user `f31954ad642e4ac0a3371f1910510b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:41:15.174976 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-f76c03ba-93b0-40ce-a4a1-93efc75ee50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:15.265735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:41:15.308102 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92337dc5-7ee3-4266-9282-79ac5941e828 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:15.311398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1167/2347] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:41:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:41:15.491684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92337dc5-7ee3-4266-9282-79ac5941e828 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:15.517880 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d7385-ce5a-4f26-ab26-ac42122aa42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:15.518771 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d7385-ce5a-4f26-ab26-ac42122aa42e admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:15.519044 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d7385-ce5a-4f26-ab26-ac42122aa42e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:15.519704 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d7385-ce5a-4f26-ab26-ac42122aa42e admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:15.520684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d7385-ce5a-4f26-ab26-ac42122aa42e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:15.534048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d7385-ce5a-4f26-ab26-ac42122aa42e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:15.595421 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1181/2348] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:41:15 2022] POST /identity/v3/projects => generated 377 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:41:15.601525 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c03ba-93b0-40ce-a4a1-93efc75ee50d None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:15.619907 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c03ba-93b0-40ce-a4a1-93efc75ee50d None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:15.766857 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbe01f-7d43-4d9f-8f32-8c97f6192b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:15.767535 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbe01f-7d43-4d9f-8f32-8c97f6192b30 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:15.767898 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbe01f-7d43-4d9f-8f32-8c97f6192b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:15.768025 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbe01f-7d43-4d9f-8f32-8c97f6192b30 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:15.768660 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccbe01f-7d43-4d9f-8f32-8c97f6192b30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:15.780733 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccbe01f-7d43-4d9f-8f32-8c97f6192b30 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:15.828838 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1168/2349] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:41:15 2022] POST /identity/v3/users => generated 289 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:41:15.832471 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d7385-ce5a-4f26-ab26-ac42122aa42e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:15.852247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d7385-ce5a-4f26-ab26-ac42122aa42e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:15.863736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af1482-c77c-4620-be18-1f3e30ba1132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:15.864801 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af1482-c77c-4620-be18-1f3e30ba1132 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:15.865303 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af1482-c77c-4620-be18-1f3e30ba1132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:15.865815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af1482-c77c-4620-be18-1f3e30ba1132 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:15.866909 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09af1482-c77c-4620-be18-1f3e30ba1132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:15.870545 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09af1482-c77c-4620-be18-1f3e30ba1132 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:15.901287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1182/2350] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:41:15.903422 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbe01f-7d43-4d9f-8f32-8c97f6192b30 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:15.928719 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbe01f-7d43-4d9f-8f32-8c97f6192b30 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:15.939495 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b597410-8363-4424-8d7b-2b332fdb95e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:15.940049 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b597410-8363-4424-8d7b-2b332fdb95e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:15.940533 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b597410-8363-4424-8d7b-2b332fdb95e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:15.940823 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b597410-8363-4424-8d7b-2b332fdb95e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:15.944347 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b597410-8363-4424-8d7b-2b332fdb95e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:15.947317 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b597410-8363-4424-8d7b-2b332fdb95e3 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:15.978489 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1169/2351] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:41:15.982889 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af1482-c77c-4620-be18-1f3e30ba1132 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:15.997125 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af1482-c77c-4620-be18-1f3e30ba1132 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:16.011683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5750a-1192-4162-8ea6-8543837debb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:16.012833 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5750a-1192-4162-8ea6-8543837debb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:16.014303 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5750a-1192-4162-8ea6-8543837debb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:16.014968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5750a-1192-4162-8ea6-8543837debb6 admin admin] PATH_INFO: `/v3/projects/6bc19b011d4e415ba126f854843b4f17/users/a566a4f86534422585b1dbc4d5ed6a1b/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:16.052859 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c5750a-1192-4162-8ea6-8543837debb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bc19b011d4e415ba126f854843b4f17, user_id=a566a4f86534422585b1dbc4d5ed6a1b, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:16.060467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c5750a-1192-4162-8ea6-8543837debb6 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:16.098035 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1183/2352] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:41:15 2022] PUT /identity/v3/projects/6bc19b011d4e415ba126f854843b4f17/users/a566a4f86534422585b1dbc4d5ed6a1b/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:41:16.099511 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea9be8-27d9-475e-a43c-f5bc8253b74a None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:16.099800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea9be8-27d9-475e-a43c-f5bc8253b74a None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:16.100141 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea9be8-27d9-475e-a43c-f5bc8253b74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:16.182733 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-8cea9be8-27d9-475e-a43c-f5bc8253b74a None None] MFA Rules not processed for user `a566a4f86534422585b1dbc4d5ed6a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:41:16.183842 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-8cea9be8-27d9-475e-a43c-f5bc8253b74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:16.406335 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1170/2353] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:41:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:41:16.408280 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5750a-1192-4162-8ea6-8543837debb6 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:16.432344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:41:16.591877 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5750a-1192-4162-8ea6-8543837debb6 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:16.604907 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994aa58-6b99-46e7-8743-03941ea5f7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:16.605682 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994aa58-6b99-46e7-8743-03941ea5f7dd admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:16.605969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994aa58-6b99-46e7-8743-03941ea5f7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:16.606231 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994aa58-6b99-46e7-8743-03941ea5f7dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:16.606909 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d994aa58-6b99-46e7-8743-03941ea5f7dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:16.608856 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d994aa58-6b99-46e7-8743-03941ea5f7dd admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:16.673754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1184/2354] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:41:16 2022] POST /identity/v3/projects => generated 377 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:41:16.674875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea9be8-27d9-475e-a43c-f5bc8253b74a None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:16.699549 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea9be8-27d9-475e-a43c-f5bc8253b74a None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:16.876384 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:41:16.891375 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf68b69-db46-4805-b957-0e9e2b23404c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:16.892295 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf68b69-db46-4805-b957-0e9e2b23404c admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:16.892772 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf68b69-db46-4805-b957-0e9e2b23404c admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:16.893393 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf68b69-db46-4805-b957-0e9e2b23404c admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:16.894403 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf68b69-db46-4805-b957-0e9e2b23404c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:16.897496 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf68b69-db46-4805-b957-0e9e2b23404c admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:16.957676 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1171/2355] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:41:16 2022] POST /identity/v3/users => generated 295 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 04:41:16.959141 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994aa58-6b99-46e7-8743-03941ea5f7dd admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:16.979134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994aa58-6b99-46e7-8743-03941ea5f7dd admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:16.998970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54911ca4-cefd-41e1-b5c3-f3dcf5016f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:16.999819 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54911ca4-cefd-41e1-b5c3-f3dcf5016f30 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:17.000883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54911ca4-cefd-41e1-b5c3-f3dcf5016f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:17.001244 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54911ca4-cefd-41e1-b5c3-f3dcf5016f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:17.002158 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54911ca4-cefd-41e1-b5c3-f3dcf5016f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:17.003860 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54911ca4-cefd-41e1-b5c3-f3dcf5016f30 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:17.034530 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf68b69-db46-4805-b957-0e9e2b23404c admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:17.041580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1185/2356] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:41:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:41:17.051718 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf68b69-db46-4805-b957-0e9e2b23404c admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:17.075665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293fdf3-9def-434e-aa6d-68e2db96fcc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:17.076618 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293fdf3-9def-434e-aa6d-68e2db96fcc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:17.077054 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293fdf3-9def-434e-aa6d-68e2db96fcc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:17.077924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293fdf3-9def-434e-aa6d-68e2db96fcc1 admin admin] PATH_INFO: `/v3/projects/28f47c9b63fa4cd3bcdab67452b167e8/users/d2e0b60c8c9e45d0805d41f066213192/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:17.138064 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293fdf3-9def-434e-aa6d-68e2db96fcc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28f47c9b63fa4cd3bcdab67452b167e8, user_id=d2e0b60c8c9e45d0805d41f066213192, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:17.140914 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293fdf3-9def-434e-aa6d-68e2db96fcc1 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:17.168368 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1172/2357] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:41:17 2022] PUT /identity/v3/projects/28f47c9b63fa4cd3bcdab67452b167e8/users/d2e0b60c8c9e45d0805d41f066213192/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:41:17.172815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54911ca4-cefd-41e1-b5c3-f3dcf5016f30 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:17.189961 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54911ca4-cefd-41e1-b5c3-f3dcf5016f30 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:17.354433 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c884a0-2a1e-46a8-928b-61d29d150fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:17.355233 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c884a0-2a1e-46a8-928b-61d29d150fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:17.355612 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c884a0-2a1e-46a8-928b-61d29d150fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:17.356007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c884a0-2a1e-46a8-928b-61d29d150fe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:17.356838 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c884a0-2a1e-46a8-928b-61d29d150fe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:17.358633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c884a0-2a1e-46a8-928b-61d29d150fe7 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:17.385848 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1186/2358] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:41:17 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:41:17.387480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293fdf3-9def-434e-aa6d-68e2db96fcc1 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:17.412500 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293fdf3-9def-434e-aa6d-68e2db96fcc1 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:17.446552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931be35e-506a-4bbd-a111-63d7b38ce6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:17.447512 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931be35e-506a-4bbd-a111-63d7b38ce6d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:17.447798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931be35e-506a-4bbd-a111-63d7b38ce6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:17.448088 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931be35e-506a-4bbd-a111-63d7b38ce6d4 admin admin] PATH_INFO: `/v3/projects/28f47c9b63fa4cd3bcdab67452b167e8/users/d2e0b60c8c9e45d0805d41f066213192/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:17.471078 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931be35e-506a-4bbd-a111-63d7b38ce6d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28f47c9b63fa4cd3bcdab67452b167e8, user_id=d2e0b60c8c9e45d0805d41f066213192, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:17.474443 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931be35e-506a-4bbd-a111-63d7b38ce6d4 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:17.535768 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1173/2359] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:41:17 2022] PUT /identity/v3/projects/28f47c9b63fa4cd3bcdab67452b167e8/users/d2e0b60c8c9e45d0805d41f066213192/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:41:17.537196 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce486b-475c-43d8-974b-9b9c3b02c1aa None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:17.543285 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce486b-475c-43d8-974b-9b9c3b02c1aa None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:17.543285 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce486b-475c-43d8-974b-9b9c3b02c1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:17.623982 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-c1ce486b-475c-43d8-974b-9b9c3b02c1aa None None] MFA Rules not processed for user `d2e0b60c8c9e45d0805d41f066213192`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:41:17.625355 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c1ce486b-475c-43d8-974b-9b9c3b02c1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:17.825580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1187/2360] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:41:17 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:41:17.835616 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931be35e-506a-4bbd-a111-63d7b38ce6d4 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:17.838621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-931be35e-506a-4bbd-a111-63d7b38ce6d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:18.038954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931be35e-506a-4bbd-a111-63d7b38ce6d4 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:18.052380 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc957216-872d-4341-8bd8-187accfc42a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:18.053158 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc957216-872d-4341-8bd8-187accfc42a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:18.053486 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc957216-872d-4341-8bd8-187accfc42a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:18.055864 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc957216-872d-4341-8bd8-187accfc42a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:18.217340 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc957216-872d-4341-8bd8-187accfc42a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:18.228054 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc957216-872d-4341-8bd8-187accfc42a6 service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:18.261560 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1174/2361] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:41:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:41:20.438638 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce486b-475c-43d8-974b-9b9c3b02c1aa None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:20.441205 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c1ce486b-475c-43d8-974b-9b9c3b02c1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:20.466164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce486b-475c-43d8-974b-9b9c3b02c1aa None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:20.477239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a71753-f910-48cc-ae4c-6c551e7b6efc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:20.477945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a71753-f910-48cc-ae4c-6c551e7b6efc service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:20.478232 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a71753-f910-48cc-ae4c-6c551e7b6efc service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:20.479600 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a71753-f910-48cc-ae4c-6c551e7b6efc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:20.509773 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:41:20.663624 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a71753-f910-48cc-ae4c-6c551e7b6efc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:20.666408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a71753-f910-48cc-ae4c-6c551e7b6efc service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:20.691107 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1188/2362] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:41:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:41:21.896321 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc957216-872d-4341-8bd8-187accfc42a6 service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:21.909882 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc957216-872d-4341-8bd8-187accfc42a6 service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:21.914437 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f4cc8-5ceb-40a1-ba48-fa71de289698 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:21.915075 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f4cc8-5ceb-40a1-ba48-fa71de289698 service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:21.915353 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f4cc8-5ceb-40a1-ba48-fa71de289698 service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:21.915628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f4cc8-5ceb-40a1-ba48-fa71de289698 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:21.926770 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f4cc8-5ceb-40a1-ba48-fa71de289698 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:21.933530 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f4cc8-5ceb-40a1-ba48-fa71de289698 service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:21.953128 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1175/2363] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:41:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:41:26.592266 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a71753-f910-48cc-ae4c-6c551e7b6efc service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:26.790906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a71753-f910-48cc-ae4c-6c551e7b6efc service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:26.799525 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64991cf7-65fc-463e-b927-8926c17e6731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:26.800290 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64991cf7-65fc-463e-b927-8926c17e6731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:26.800811 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64991cf7-65fc-463e-b927-8926c17e6731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:26.801151 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64991cf7-65fc-463e-b927-8926c17e6731 admin admin] PATH_INFO: `/v3/users/f31954ad642e4ac0a3371f1910510b7a` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:26.802479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64991cf7-65fc-463e-b927-8926c17e6731 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f31954ad642e4ac0a3371f1910510b7a)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:26.805475 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64991cf7-65fc-463e-b927-8926c17e6731 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:26.884534 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-64991cf7-65fc-463e-b927-8926c17e6731 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f31954ad642e4ac0a3371f1910510b7a'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:26.922556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-64991cf7-65fc-463e-b927-8926c17e6731 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f31954ad642e4ac0a3371f1910510b7a'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:26.934995 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-64991cf7-65fc-463e-b927-8926c17e6731 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f31954ad642e4ac0a3371f1910510b7a'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:27.027446 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-64991cf7-65fc-463e-b927-8926c17e6731 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f31954ad642e4ac0a3371f1910510b7a'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:27.031249 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-64991cf7-65fc-463e-b927-8926c17e6731 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f31954ad642e4ac0a3371f1910510b7a'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:27.072106 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1189/2364] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:41:26 2022] DELETE /identity/v3/users/f31954ad642e4ac0a3371f1910510b7a => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:41:27.074764 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d17155-9abf-4873-b70c-3a40819f5efd None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:27.075317 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d17155-9abf-4873-b70c-3a40819f5efd None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:27.075849 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d17155-9abf-4873-b70c-3a40819f5efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:27.132895 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-10d17155-9abf-4873-b70c-3a40819f5efd None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:41:27.135404 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-10d17155-9abf-4873-b70c-3a40819f5efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:27.298243 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:41:27.330507 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1176/2365] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:41:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:41:27.361243 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64991cf7-65fc-463e-b927-8926c17e6731 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:27.364238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-64991cf7-65fc-463e-b927-8926c17e6731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:27.454764 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:41:27.556064 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64991cf7-65fc-463e-b927-8926c17e6731 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:27.579775 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882cd604-b34f-4bb8-be56-ae65211ed9af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:27.580573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cd604-b34f-4bb8-be56-ae65211ed9af service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:27.581039 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cd604-b34f-4bb8-be56-ae65211ed9af service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:27.581399 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cd604-b34f-4bb8-be56-ae65211ed9af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:27.609263 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882cd604-b34f-4bb8-be56-ae65211ed9af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:27.613067 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882cd604-b34f-4bb8-be56-ae65211ed9af service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:27.649690 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1190/2366] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:41:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:41:28.683516 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d17155-9abf-4873-b70c-3a40819f5efd None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:28.685024 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-10d17155-9abf-4873-b70c-3a40819f5efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:28.720197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d17155-9abf-4873-b70c-3a40819f5efd None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:28.760206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4881d3-6c01-42cf-90b1-cf0fc8573fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:28.760963 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4881d3-6c01-42cf-90b1-cf0fc8573fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:28.761253 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4881d3-6c01-42cf-90b1-cf0fc8573fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:28.761600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4881d3-6c01-42cf-90b1-cf0fc8573fd0 admin admin] PATH_INFO: `/v3/projects/f775bbf7216a4717bc9163cedb31b7cf` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:28.763568 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4881d3-6c01-42cf-90b1-cf0fc8573fd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f775bbf7216a4717bc9163cedb31b7cf)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:28.766033 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4881d3-6c01-42cf-90b1-cf0fc8573fd0 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:28.892202 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-db4881d3-6c01-42cf-90b1-cf0fc8573fd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f775bbf7216a4717bc9163cedb31b7cf', 'request_id': 'req-db4881d3-6c01-42cf-90b1-cf0fc8573fd0'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:28.907533 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-db4881d3-6c01-42cf-90b1-cf0fc8573fd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f775bbf7216a4717bc9163cedb31b7cf', 'request_id': 'req-db4881d3-6c01-42cf-90b1-cf0fc8573fd0'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:28.919025 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-db4881d3-6c01-42cf-90b1-cf0fc8573fd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f775bbf7216a4717bc9163cedb31b7cf', 'request_id': 'req-db4881d3-6c01-42cf-90b1-cf0fc8573fd0'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:28.983894 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-db4881d3-6c01-42cf-90b1-cf0fc8573fd0 admin admin] The token cache is being invalidate because project f775bbf7216a4717bc9163cedb31b7cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:41:28.987685 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-db4881d3-6c01-42cf-90b1-cf0fc8573fd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:28.991655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-db4881d3-6c01-42cf-90b1-cf0fc8573fd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:28.998716 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1177/2367] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:41:28 2022] DELETE /identity/v3/projects/f775bbf7216a4717bc9163cedb31b7cf => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:41:29.002821 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882cd604-b34f-4bb8-be56-ae65211ed9af service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:29.017025 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882cd604-b34f-4bb8-be56-ae65211ed9af service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:29.224866 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fde7ae-d032-4720-9f9c-c9f2a47d9f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:29.226820 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fde7ae-d032-4720-9f9c-c9f2a47d9f2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:29.227183 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fde7ae-d032-4720-9f9c-c9f2a47d9f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:29.227535 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fde7ae-d032-4720-9f9c-c9f2a47d9f2a admin admin] PATH_INFO: `/v3/users/a566a4f86534422585b1dbc4d5ed6a1b` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:29.229822 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fde7ae-d032-4720-9f9c-c9f2a47d9f2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a566a4f86534422585b1dbc4d5ed6a1b)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:29.232461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fde7ae-d032-4720-9f9c-c9f2a47d9f2a admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:29.346001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e2fde7ae-d032-4720-9f9c-c9f2a47d9f2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a566a4f86534422585b1dbc4d5ed6a1b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:29.363124 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e2fde7ae-d032-4720-9f9c-c9f2a47d9f2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a566a4f86534422585b1dbc4d5ed6a1b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:29.365964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e2fde7ae-d032-4720-9f9c-c9f2a47d9f2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a566a4f86534422585b1dbc4d5ed6a1b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:29.381974 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e2fde7ae-d032-4720-9f9c-c9f2a47d9f2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a566a4f86534422585b1dbc4d5ed6a1b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:29.382731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e2fde7ae-d032-4720-9f9c-c9f2a47d9f2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a566a4f86534422585b1dbc4d5ed6a1b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:29.429580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1191/2368] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:41:29 2022] DELETE /identity/v3/users/a566a4f86534422585b1dbc4d5ed6a1b => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:41:29.934293 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4881d3-6c01-42cf-90b1-cf0fc8573fd0 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:29.937130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-db4881d3-6c01-42cf-90b1-cf0fc8573fd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:29.974504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4881d3-6c01-42cf-90b1-cf0fc8573fd0 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:30.097636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4818654-a5ec-4542-be70-22f89e09211d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:30.098293 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4818654-a5ec-4542-be70-22f89e09211d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:30.098585 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4818654-a5ec-4542-be70-22f89e09211d admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:30.098863 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4818654-a5ec-4542-be70-22f89e09211d admin admin] PATH_INFO: `/v3/projects/6bc19b011d4e415ba126f854843b4f17` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:30.100121 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4818654-a5ec-4542-be70-22f89e09211d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bc19b011d4e415ba126f854843b4f17)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:30.102456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4818654-a5ec-4542-be70-22f89e09211d admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:30.166218 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c4818654-a5ec-4542-be70-22f89e09211d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bc19b011d4e415ba126f854843b4f17', 'request_id': 'req-c4818654-a5ec-4542-be70-22f89e09211d'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:30.174120 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c4818654-a5ec-4542-be70-22f89e09211d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bc19b011d4e415ba126f854843b4f17', 'request_id': 'req-c4818654-a5ec-4542-be70-22f89e09211d'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:30.180239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c4818654-a5ec-4542-be70-22f89e09211d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bc19b011d4e415ba126f854843b4f17', 'request_id': 'req-c4818654-a5ec-4542-be70-22f89e09211d'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:30.191484 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c4818654-a5ec-4542-be70-22f89e09211d admin admin] The token cache is being invalidate because project 6bc19b011d4e415ba126f854843b4f17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:41:30.191771 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c4818654-a5ec-4542-be70-22f89e09211d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:30.192596 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c4818654-a5ec-4542-be70-22f89e09211d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:30.195386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1178/2369] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:41:29 2022] DELETE /identity/v3/projects/6bc19b011d4e415ba126f854843b4f17 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:41:30.199545 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fde7ae-d032-4720-9f9c-c9f2a47d9f2a admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:30.200958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e2fde7ae-d032-4720-9f9c-c9f2a47d9f2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:30.216887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fde7ae-d032-4720-9f9c-c9f2a47d9f2a admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:30.262090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:41:30.276461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951f09d-e73d-4a54-bf93-358049787303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:30.277116 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951f09d-e73d-4a54-bf93-358049787303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:30.277421 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951f09d-e73d-4a54-bf93-358049787303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:30.277793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951f09d-e73d-4a54-bf93-358049787303 admin admin] PATH_INFO: `/v3/users/d2e0b60c8c9e45d0805d41f066213192` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:30.279085 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a951f09d-e73d-4a54-bf93-358049787303 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2e0b60c8c9e45d0805d41f066213192)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:30.281280 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a951f09d-e73d-4a54-bf93-358049787303 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:30.330926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a951f09d-e73d-4a54-bf93-358049787303 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2e0b60c8c9e45d0805d41f066213192'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:30.338833 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a951f09d-e73d-4a54-bf93-358049787303 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2e0b60c8c9e45d0805d41f066213192'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:30.339567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a951f09d-e73d-4a54-bf93-358049787303 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2e0b60c8c9e45d0805d41f066213192'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:30.351945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a951f09d-e73d-4a54-bf93-358049787303 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2e0b60c8c9e45d0805d41f066213192'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:30.352654 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a951f09d-e73d-4a54-bf93-358049787303 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2e0b60c8c9e45d0805d41f066213192'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:30.366269 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1192/2370] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:41:30 2022] DELETE /identity/v3/users/d2e0b60c8c9e45d0805d41f066213192 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:41:30.657703 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4818654-a5ec-4542-be70-22f89e09211d admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:30.659219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c4818654-a5ec-4542-be70-22f89e09211d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:30.669495 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:41:30.675798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4818654-a5ec-4542-be70-22f89e09211d admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:30.744121 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002aa83-c232-48c7-abc6-1f5303811486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:30.744676 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002aa83-c232-48c7-abc6-1f5303811486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:30.744956 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002aa83-c232-48c7-abc6-1f5303811486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:30.745228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002aa83-c232-48c7-abc6-1f5303811486 admin admin] PATH_INFO: `/v3/projects/28f47c9b63fa4cd3bcdab67452b167e8` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:30.746576 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a002aa83-c232-48c7-abc6-1f5303811486 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28f47c9b63fa4cd3bcdab67452b167e8)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:30.748470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a002aa83-c232-48c7-abc6-1f5303811486 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:30.796290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a002aa83-c232-48c7-abc6-1f5303811486 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28f47c9b63fa4cd3bcdab67452b167e8', 'request_id': 'req-a002aa83-c232-48c7-abc6-1f5303811486'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:30.806025 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a002aa83-c232-48c7-abc6-1f5303811486 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28f47c9b63fa4cd3bcdab67452b167e8', 'request_id': 'req-a002aa83-c232-48c7-abc6-1f5303811486'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:30.813314 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a002aa83-c232-48c7-abc6-1f5303811486 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28f47c9b63fa4cd3bcdab67452b167e8', 'request_id': 'req-a002aa83-c232-48c7-abc6-1f5303811486'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:30.826232 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a002aa83-c232-48c7-abc6-1f5303811486 admin admin] The token cache is being invalidate because project 28f47c9b63fa4cd3bcdab67452b167e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:41:30.826522 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a002aa83-c232-48c7-abc6-1f5303811486 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:30.827199 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a002aa83-c232-48c7-abc6-1f5303811486 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:30.828773 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1179/2371] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:41:30 2022] DELETE /identity/v3/projects/28f47c9b63fa4cd3bcdab67452b167e8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:41:30.833946 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e8741-3a13-4fce-a698-117757938c7b None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:30.834239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e8741-3a13-4fce-a698-117757938c7b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:30.834968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e8741-3a13-4fce-a698-117757938c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:30.855537 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-f75e8741-3a13-4fce-a698-117757938c7b None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:41:30.856737 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f75e8741-3a13-4fce-a698-117757938c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:30.920165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1193/2372] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:41:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:41:30.928083 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de545d7c-0297-4def-9212-5ce3eaaa2c30 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:30.928515 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de545d7c-0297-4def-9212-5ce3eaaa2c30 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:30.928774 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de545d7c-0297-4def-9212-5ce3eaaa2c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:30.949867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-de545d7c-0297-4def-9212-5ce3eaaa2c30 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:41:30.951106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-de545d7c-0297-4def-9212-5ce3eaaa2c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:30.960741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1180/2373] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:41:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:41:30.965229 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e8741-3a13-4fce-a698-117757938c7b None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:30.976358 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e8741-3a13-4fce-a698-117757938c7b None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:30.980986 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51fa48f-f9e0-44c5-b6ad-035916462940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:30.981674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fa48f-f9e0-44c5-b6ad-035916462940 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:30.981988 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fa48f-f9e0-44c5-b6ad-035916462940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:30.982203 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fa48f-f9e0-44c5-b6ad-035916462940 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:30.982929 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51fa48f-f9e0-44c5-b6ad-035916462940 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:41:30.983454 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51fa48f-f9e0-44c5-b6ad-035916462940 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:30.985041 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51fa48f-f9e0-44c5-b6ad-035916462940 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:30.997966 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1194/2374] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:41:31.002296 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de545d7c-0297-4def-9212-5ce3eaaa2c30 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:31.014688 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de545d7c-0297-4def-9212-5ce3eaaa2c30 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:31.026254 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98a00e-4710-4d1e-8684-e6e25e95c75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:31.026914 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98a00e-4710-4d1e-8684-e6e25e95c75d admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:31.027207 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98a00e-4710-4d1e-8684-e6e25e95c75d admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:31.027488 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98a00e-4710-4d1e-8684-e6e25e95c75d admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:31.028178 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b98a00e-4710-4d1e-8684-e6e25e95c75d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:31.030707 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b98a00e-4710-4d1e-8684-e6e25e95c75d admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:31.074146 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1181/2375] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:41:31 2022] POST /identity/v3/projects => generated 369 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:41:31.077977 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51fa48f-f9e0-44c5-b6ad-035916462940 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:31.089350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51fa48f-f9e0-44c5-b6ad-035916462940 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:31.104053 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3562bf1-64d3-45a9-b292-a4a62b1a71e1 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:31.104340 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3562bf1-64d3-45a9-b292-a4a62b1a71e1 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:31.104614 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3562bf1-64d3-45a9-b292-a4a62b1a71e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:31.127140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-f3562bf1-64d3-45a9-b292-a4a62b1a71e1 None None] MFA Rules not processed for user `3a721708c7e443e799349add27de8f84`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:41:31.128456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-f3562bf1-64d3-45a9-b292-a4a62b1a71e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:31.143866 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:41:31.155262 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb2396-1241-4172-bd19-39ed0b5df3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:31.155894 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb2396-1241-4172-bd19-39ed0b5df3d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:31.156429 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb2396-1241-4172-bd19-39ed0b5df3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:31.156796 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb2396-1241-4172-bd19-39ed0b5df3d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:31.157834 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb2396-1241-4172-bd19-39ed0b5df3d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:31.160715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb2396-1241-4172-bd19-39ed0b5df3d4 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:31.187741 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1195/2376] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:41:31 2022] POST /identity/v3/users => generated 285 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:41:31.191407 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb2396-1241-4172-bd19-39ed0b5df3d4 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:31.198109 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1182/2377] 10.222.0.47 () {58 vars in 967 bytes} [Thu Aug 4 04:41:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:41:31.203056 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb2396-1241-4172-bd19-39ed0b5df3d4 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:31.215061 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3562bf1-64d3-45a9-b292-a4a62b1a71e1 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:31.216482 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-f3562bf1-64d3-45a9-b292-a4a62b1a71e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:31.220118 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407fca5-bd13-413b-928c-413999d41f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:31.220118 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407fca5-bd13-413b-928c-413999d41f11 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:31.220118 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407fca5-bd13-413b-928c-413999d41f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:31.220118 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407fca5-bd13-413b-928c-413999d41f11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:31.220504 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407fca5-bd13-413b-928c-413999d41f11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:31.221838 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407fca5-bd13-413b-928c-413999d41f11 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:31.229097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:41:31.237587 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1196/2378] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:41:31.241843 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407fca5-bd13-413b-928c-413999d41f11 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:31.252720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407fca5-bd13-413b-928c-413999d41f11 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:31.259645 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcbc7f-37c2-47b8-940d-6d3f581ecc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:31.260727 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcbc7f-37c2-47b8-940d-6d3f581ecc3e admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:31.261242 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcbc7f-37c2-47b8-940d-6d3f581ecc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:31.261797 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcbc7f-37c2-47b8-940d-6d3f581ecc3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:31.263149 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bcbc7f-37c2-47b8-940d-6d3f581ecc3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:31.264913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bcbc7f-37c2-47b8-940d-6d3f581ecc3e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:31.280204 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1197/2379] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:41:31.282009 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcbc7f-37c2-47b8-940d-6d3f581ecc3e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:31.294326 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcbc7f-37c2-47b8-940d-6d3f581ecc3e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:31.299260 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e75fb-3146-4a83-9fc2-fc2e1f9e4f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:31.299930 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e75fb-3146-4a83-9fc2-fc2e1f9e4f8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:31.300214 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e75fb-3146-4a83-9fc2-fc2e1f9e4f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:31.300604 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e75fb-3146-4a83-9fc2-fc2e1f9e4f8c admin admin] PATH_INFO: `/v3/projects/59c77f4178e3413896315375759e337a/users/eb9f52a82475425dba0fcabf50ff43f5/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:31.307514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3562bf1-64d3-45a9-b292-a4a62b1a71e1 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:31.313374 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a7e278-dadb-4109-834a-301f10f33617 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1e539006166434790823b839bd745d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:31.314174 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7e278-dadb-4109-834a-301f10f33617 service swift] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:31.314798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7e278-dadb-4109-834a-301f10f33617 service swift] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:31.315084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7e278-dadb-4109-834a-301f10f33617 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:31.319658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e75fb-3146-4a83-9fc2-fc2e1f9e4f8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=59c77f4178e3413896315375759e337a, user_id=eb9f52a82475425dba0fcabf50ff43f5, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:31.322408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e75fb-3146-4a83-9fc2-fc2e1f9e4f8c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:31.324225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a7e278-dadb-4109-834a-301f10f33617 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:31.326003 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a7e278-dadb-4109-834a-301f10f33617 service swift] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:31.336200 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1183/2380] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Aug 4 04:41:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 04:41:31.338829 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1198/2381] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:41:31 2022] PUT /identity/v3/projects/59c77f4178e3413896315375759e337a/users/eb9f52a82475425dba0fcabf50ff43f5/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:41:31.349220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56093831-7707-4f36-8456-7e8b19d56ce6 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:31.349512 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56093831-7707-4f36-8456-7e8b19d56ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:31.349821 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56093831-7707-4f36-8456-7e8b19d56ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:31.387397 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-56093831-7707-4f36-8456-7e8b19d56ce6 None None] MFA Rules not processed for user `eb9f52a82475425dba0fcabf50ff43f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:41:31.388929 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-56093831-7707-4f36-8456-7e8b19d56ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:31.463010 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1184/2382] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:41:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:41:31.475054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e75fb-3146-4a83-9fc2-fc2e1f9e4f8c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:31.477505 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0b7e75fb-3146-4a83-9fc2-fc2e1f9e4f8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:31.544916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e75fb-3146-4a83-9fc2-fc2e1f9e4f8c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:31.548855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205e523-ca1a-4ad0-b489-de38edec8503 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b77a955866c54a43a7aaa271fefdec57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:31.549524 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205e523-ca1a-4ad0-b489-de38edec8503 service glance] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:31.549868 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205e523-ca1a-4ad0-b489-de38edec8503 service glance] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:31.550140 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205e523-ca1a-4ad0-b489-de38edec8503 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:31.559476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6205e523-ca1a-4ad0-b489-de38edec8503 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:31.560905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6205e523-ca1a-4ad0-b489-de38edec8503 service glance] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:31.573007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1199/2383] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:41:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:41:33.459654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56093831-7707-4f36-8456-7e8b19d56ce6 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:33.461494 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-56093831-7707-4f36-8456-7e8b19d56ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:33.571298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56093831-7707-4f36-8456-7e8b19d56ce6 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:33.578269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8428991-0055-42de-b6af-4f9160d05dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:33.579480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8428991-0055-42de-b6af-4f9160d05dbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:33.579698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8428991-0055-42de-b6af-4f9160d05dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:33.579978 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8428991-0055-42de-b6af-4f9160d05dbc admin admin] PATH_INFO: `/v3/users/1b23dd1311554910a19399aa74df9e4c` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:33.581496 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8428991-0055-42de-b6af-4f9160d05dbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b23dd1311554910a19399aa74df9e4c)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:33.583480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8428991-0055-42de-b6af-4f9160d05dbc admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:33.627907 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:41:33.648290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-b8428991-0055-42de-b6af-4f9160d05dbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b23dd1311554910a19399aa74df9e4c'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:33.649302 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-b8428991-0055-42de-b6af-4f9160d05dbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b23dd1311554910a19399aa74df9e4c'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:33.666594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-b8428991-0055-42de-b6af-4f9160d05dbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b23dd1311554910a19399aa74df9e4c'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:33.676594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-b8428991-0055-42de-b6af-4f9160d05dbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b23dd1311554910a19399aa74df9e4c'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:33.677813 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-b8428991-0055-42de-b6af-4f9160d05dbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b23dd1311554910a19399aa74df9e4c'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:33.698962 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1185/2384] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:41:33 2022] DELETE /identity/v3/users/1b23dd1311554910a19399aa74df9e4c => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:41:33.882171 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205e523-ca1a-4ad0-b489-de38edec8503 service glance] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:33.893735 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205e523-ca1a-4ad0-b489-de38edec8503 service glance] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:33.955803 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4ba0e-0dd5-421d-9bc8-ed864dbf2d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:33.956440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4ba0e-0dd5-421d-9bc8-ed864dbf2d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:33.956718 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4ba0e-0dd5-421d-9bc8-ed864dbf2d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:33.956988 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4ba0e-0dd5-421d-9bc8-ed864dbf2d75 admin admin] PATH_INFO: `/v3/projects/23ec86581683456b8c52c649eed292e7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:33.958161 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e4ba0e-0dd5-421d-9bc8-ed864dbf2d75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23ec86581683456b8c52c649eed292e7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:33.960651 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e4ba0e-0dd5-421d-9bc8-ed864dbf2d75 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:34.009357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:41:34.013401 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-61e4ba0e-0dd5-421d-9bc8-ed864dbf2d75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23ec86581683456b8c52c649eed292e7', 'request_id': 'req-61e4ba0e-0dd5-421d-9bc8-ed864dbf2d75'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:34.021376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-61e4ba0e-0dd5-421d-9bc8-ed864dbf2d75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23ec86581683456b8c52c649eed292e7', 'request_id': 'req-61e4ba0e-0dd5-421d-9bc8-ed864dbf2d75'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:34.028038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-61e4ba0e-0dd5-421d-9bc8-ed864dbf2d75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23ec86581683456b8c52c649eed292e7', 'request_id': 'req-61e4ba0e-0dd5-421d-9bc8-ed864dbf2d75'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:34.039581 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-61e4ba0e-0dd5-421d-9bc8-ed864dbf2d75 admin admin] The token cache is being invalidate because project 23ec86581683456b8c52c649eed292e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:41:34.039866 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-61e4ba0e-0dd5-421d-9bc8-ed864dbf2d75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:34.040523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-61e4ba0e-0dd5-421d-9bc8-ed864dbf2d75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:41:34.042300 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1200/2385] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:41:33 2022] DELETE /identity/v3/projects/23ec86581683456b8c52c649eed292e7 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:41:34.049581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8b8bb-a06d-42ad-91f4-93def21c268a None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:34.049581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8b8bb-a06d-42ad-91f4-93def21c268a None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:34.049581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8b8bb-a06d-42ad-91f4-93def21c268a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:34.070678 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-c4a8b8bb-a06d-42ad-91f4-93def21c268a None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:41:34.071893 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c4a8b8bb-a06d-42ad-91f4-93def21c268a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:34.171977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1186/2386] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:41:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:41:34.174494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ae7ba-eafe-4794-ba47-f0f466e298b1 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:34.174995 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ae7ba-eafe-4794-ba47-f0f466e298b1 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:34.175514 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ae7ba-eafe-4794-ba47-f0f466e298b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:34.197788 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-d93ae7ba-eafe-4794-ba47-f0f466e298b1 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:41:34.199534 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d93ae7ba-eafe-4794-ba47-f0f466e298b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:34.213029 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1201/2387] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:41:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:41:34.214020 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8b8bb-a06d-42ad-91f4-93def21c268a None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:34.224617 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8b8bb-a06d-42ad-91f4-93def21c268a None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:34.230335 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496620ce-abfc-4c87-82b7-d590806a04bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:34.230973 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496620ce-abfc-4c87-82b7-d590806a04bc admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:34.231255 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496620ce-abfc-4c87-82b7-d590806a04bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:34.231528 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496620ce-abfc-4c87-82b7-d590806a04bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:34.232288 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496620ce-abfc-4c87-82b7-d590806a04bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:41:34.232596 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496620ce-abfc-4c87-82b7-d590806a04bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:34.234257 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496620ce-abfc-4c87-82b7-d590806a04bc admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:34.257602 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1187/2388] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:41:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 04:41:34.262916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ae7ba-eafe-4794-ba47-f0f466e298b1 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:34.273400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ae7ba-eafe-4794-ba47-f0f466e298b1 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:34.284188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dccf81a-d4c0-482c-91e8-014704894a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:34.284822 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dccf81a-d4c0-482c-91e8-014704894a24 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:34.285101 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dccf81a-d4c0-482c-91e8-014704894a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:34.285372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dccf81a-d4c0-482c-91e8-014704894a24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:34.286105 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dccf81a-d4c0-482c-91e8-014704894a24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:34.288246 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dccf81a-d4c0-482c-91e8-014704894a24 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:34.325614 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1202/2389] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:41:34 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:41:34.330010 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496620ce-abfc-4c87-82b7-d590806a04bc admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:34.343703 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496620ce-abfc-4c87-82b7-d590806a04bc admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:34.414990 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e1d22-1176-4217-8f1c-2e9519384fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:34.415643 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e1d22-1176-4217-8f1c-2e9519384fbc admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:34.415930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e1d22-1176-4217-8f1c-2e9519384fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:34.416206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e1d22-1176-4217-8f1c-2e9519384fbc admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:34.416888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e1d22-1176-4217-8f1c-2e9519384fbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:34.419274 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e1d22-1176-4217-8f1c-2e9519384fbc admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:34.442137 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1188/2390] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:41:34 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:41:34.446833 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dccf81a-d4c0-482c-91e8-014704894a24 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:34.458813 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dccf81a-d4c0-482c-91e8-014704894a24 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:34.466615 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ef778-ed0f-45ce-ada9-c5bb4aba2f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:34.467287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ef778-ed0f-45ce-ada9-c5bb4aba2f62 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:34.467603 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ef778-ed0f-45ce-ada9-c5bb4aba2f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:34.467910 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ef778-ed0f-45ce-ada9-c5bb4aba2f62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:34.468788 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ef778-ed0f-45ce-ada9-c5bb4aba2f62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:34.470857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ef778-ed0f-45ce-ada9-c5bb4aba2f62 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:34.483286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1203/2391] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 04:41:34.487699 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e1d22-1176-4217-8f1c-2e9519384fbc admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:34.498019 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e1d22-1176-4217-8f1c-2e9519384fbc admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:34.505905 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b372dc-a3a7-4dc0-aae0-159d4001adc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:34.507117 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b372dc-a3a7-4dc0-aae0-159d4001adc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:34.507402 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b372dc-a3a7-4dc0-aae0-159d4001adc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:34.507668 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b372dc-a3a7-4dc0-aae0-159d4001adc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:34.508375 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b372dc-a3a7-4dc0-aae0-159d4001adc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:34.510110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b372dc-a3a7-4dc0-aae0-159d4001adc6 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:34.525332 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1189/2392] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:41:34.526645 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ef778-ed0f-45ce-ada9-c5bb4aba2f62 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:34.537130 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ef778-ed0f-45ce-ada9-c5bb4aba2f62 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:34.542390 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeeaef6-2bf3-4274-8f26-d79706aa535a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:34.543056 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeeaef6-2bf3-4274-8f26-d79706aa535a admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:34.543345 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeeaef6-2bf3-4274-8f26-d79706aa535a admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:34.543622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeeaef6-2bf3-4274-8f26-d79706aa535a admin admin] PATH_INFO: `/v3/projects/58d8f4b47019401ab6e953bbbbd801fd/users/d604afd3c4234473ac1f2fcff2501743/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:34.560393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeeaef6-2bf3-4274-8f26-d79706aa535a admin admin] RBAC: Authorizing `identity:create_grant(project_id=58d8f4b47019401ab6e953bbbbd801fd, user_id=d604afd3c4234473ac1f2fcff2501743, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:34.564322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeeaef6-2bf3-4274-8f26-d79706aa535a admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:34.578677 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1204/2393] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:41:34 2022] PUT /identity/v3/projects/58d8f4b47019401ab6e953bbbbd801fd/users/d604afd3c4234473ac1f2fcff2501743/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:41:34.583814 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b931ff-6676-48fa-9eda-90041b88f400 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:34.584142 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b931ff-6676-48fa-9eda-90041b88f400 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:34.585137 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b931ff-6676-48fa-9eda-90041b88f400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:34.605201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-36b931ff-6676-48fa-9eda-90041b88f400 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:41:34.606706 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-36b931ff-6676-48fa-9eda-90041b88f400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:34.618824 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:41:34.684733 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1190/2394] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:41:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:41:34.695018 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeeaef6-2bf3-4274-8f26-d79706aa535a admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:34.696747 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6aeeaef6-2bf3-4274-8f26-d79706aa535a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:34.758525 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeeaef6-2bf3-4274-8f26-d79706aa535a admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:34.762221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e6ad7-da67-4caf-b508-aa4f6a38cc1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:34.762839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e6ad7-da67-4caf-b508-aa4f6a38cc1e service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:34.763116 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e6ad7-da67-4caf-b508-aa4f6a38cc1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:34.763386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e6ad7-da67-4caf-b508-aa4f6a38cc1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:34.772963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888e6ad7-da67-4caf-b508-aa4f6a38cc1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:34.774415 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888e6ad7-da67-4caf-b508-aa4f6a38cc1e service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:34.784059 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1205/2395] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:41:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:41:35.534272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f9855-fbc1-4541-b085-60d7042ddea8 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:35.534812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f9855-fbc1-4541-b085-60d7042ddea8 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:35.534812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f9855-fbc1-4541-b085-60d7042ddea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:35.566404 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-c26f9855-fbc1-4541-b085-60d7042ddea8 None None] MFA Rules not processed for user `d604afd3c4234473ac1f2fcff2501743`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:41:35.567644 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c26f9855-fbc1-4541-b085-60d7042ddea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:35.647301 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1191/2396] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:41:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:41:35.658423 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e6ad7-da67-4caf-b508-aa4f6a38cc1e service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:35.659915 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-888e6ad7-da67-4caf-b508-aa4f6a38cc1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:35.726977 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e6ad7-da67-4caf-b508-aa4f6a38cc1e service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:35.732113 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd74fc5-5a2a-4625-9e08-adc96de85e43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:35.733166 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd74fc5-5a2a-4625-9e08-adc96de85e43 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:35.733326 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd74fc5-5a2a-4625-9e08-adc96de85e43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:35.733505 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd74fc5-5a2a-4625-9e08-adc96de85e43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:35.748976 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd74fc5-5a2a-4625-9e08-adc96de85e43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:35.750737 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd74fc5-5a2a-4625-9e08-adc96de85e43 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:35.763435 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1206/2397] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:41:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:41:37.313288 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cb410-c9d8-4aff-8f18-1dc5c0fe8ca1 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:37.313900 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cb410-c9d8-4aff-8f18-1dc5c0fe8ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:37.314297 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cb410-c9d8-4aff-8f18-1dc5c0fe8ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:37.338078 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-5a1cb410-c9d8-4aff-8f18-1dc5c0fe8ca1 None None] MFA Rules not processed for user `d604afd3c4234473ac1f2fcff2501743`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:41:37.339537 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5a1cb410-c9d8-4aff-8f18-1dc5c0fe8ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:37.356238 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1192/2398] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:41:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:41:45.758094 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=87,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:41:47.331042 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=56 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:41:51.892514 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd74fc5-5a2a-4625-9e08-adc96de85e43 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:51.903096 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd74fc5-5a2a-4625-9e08-adc96de85e43 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:51.907169 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b24a5-eae7-4185-b0fc-9927cf4279fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:51.907857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b24a5-eae7-4185-b0fc-9927cf4279fd service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:51.908211 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b24a5-eae7-4185-b0fc-9927cf4279fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:51.908557 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b24a5-eae7-4185-b0fc-9927cf4279fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:51.910408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-418b24a5-eae7-4185-b0fc-9927cf4279fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:51.976925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b24a5-eae7-4185-b0fc-9927cf4279fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:51.978269 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b24a5-eae7-4185-b0fc-9927cf4279fd service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:51.988067 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1207/2399] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:41:51 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:41:58.985344 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1cb410-c9d8-4aff-8f18-1dc5c0fe8ca1 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:41:58.987756 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5a1cb410-c9d8-4aff-8f18-1dc5c0fe8ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:59.099862 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1cb410-c9d8-4aff-8f18-1dc5c0fe8ca1 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:41:59.105321 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728765c1-7a60-4579-b8bf-15b1ce0a4b0f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350f72f740f749f68175c1c4e8ae9d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:41:59.105982 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728765c1-7a60-4579-b8bf-15b1ce0a4b0f service placement] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:41:59.106373 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728765c1-7a60-4579-b8bf-15b1ce0a4b0f service placement] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:41:59.106860 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728765c1-7a60-4579-b8bf-15b1ce0a4b0f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:41:59.108670 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-728765c1-7a60-4579-b8bf-15b1ce0a4b0f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:41:59.126417 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728765c1-7a60-4579-b8bf-15b1ce0a4b0f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:41:59.128088 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728765c1-7a60-4579-b8bf-15b1ce0a4b0f service placement] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:41:59.141755 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1193/2400] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:41:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:42:01.983415 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=30 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:42:08.609662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b24a5-eae7-4185-b0fc-9927cf4279fd service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:42:08.611197 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-418b24a5-eae7-4185-b0fc-9927cf4279fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:42:08.626295 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b24a5-eae7-4185-b0fc-9927cf4279fd service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:42:08.632172 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b9491-9a0f-4b00-8cc6-bd13b08ff0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:42:08.633012 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b9491-9a0f-4b00-8cc6-bd13b08ff0af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:08.633336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b9491-9a0f-4b00-8cc6-bd13b08ff0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:08.633639 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b9491-9a0f-4b00-8cc6-bd13b08ff0af admin admin] PATH_INFO: `/v3/users/eb9f52a82475425dba0fcabf50ff43f5` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:08.634890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b9491-9a0f-4b00-8cc6-bd13b08ff0af admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb9f52a82475425dba0fcabf50ff43f5)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:42:08.636930 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b9491-9a0f-4b00-8cc6-bd13b08ff0af admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:42:08.685847 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-874b9491-9a0f-4b00-8cc6-bd13b08ff0af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb9f52a82475425dba0fcabf50ff43f5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:42:08.693343 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-874b9491-9a0f-4b00-8cc6-bd13b08ff0af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb9f52a82475425dba0fcabf50ff43f5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:42:08.694113 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-874b9491-9a0f-4b00-8cc6-bd13b08ff0af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb9f52a82475425dba0fcabf50ff43f5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:42:08.706763 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-874b9491-9a0f-4b00-8cc6-bd13b08ff0af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb9f52a82475425dba0fcabf50ff43f5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:42:08.707538 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-874b9491-9a0f-4b00-8cc6-bd13b08ff0af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb9f52a82475425dba0fcabf50ff43f5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:42:08.721221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1208/2401] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:42:08 2022] DELETE /identity/v3/users/eb9f52a82475425dba0fcabf50ff43f5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:42:08.726387 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119422a-a44a-4ad9-a234-1eb9e5f94ec0 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:08.726911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119422a-a44a-4ad9-a234-1eb9e5f94ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:08.727062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119422a-a44a-4ad9-a234-1eb9e5f94ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:08.747931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-7119422a-a44a-4ad9-a234-1eb9e5f94ec0 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:42:08.749145 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-7119422a-a44a-4ad9-a234-1eb9e5f94ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:42:08.815642 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1194/2402] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:42:08.826497 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b9491-9a0f-4b00-8cc6-bd13b08ff0af admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:42:08.827940 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-874b9491-9a0f-4b00-8cc6-bd13b08ff0af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:42:08.891165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b9491-9a0f-4b00-8cc6-bd13b08ff0af admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:42:08.894770 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f6580-bbdd-40ca-941d-9e614537941b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:42:08.895393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f6580-bbdd-40ca-941d-9e614537941b service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:08.895669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f6580-bbdd-40ca-941d-9e614537941b service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:08.895940 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f6580-bbdd-40ca-941d-9e614537941b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:08.906840 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f6580-bbdd-40ca-941d-9e614537941b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:42:08.908272 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f6580-bbdd-40ca-941d-9e614537941b service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:42:08.918724 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1209/2403] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:42:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:42:09.198572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119422a-a44a-4ad9-a234-1eb9e5f94ec0 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:42:09.200255 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-7119422a-a44a-4ad9-a234-1eb9e5f94ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:42:09.213193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119422a-a44a-4ad9-a234-1eb9e5f94ec0 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:42:09.219390 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e27de5-a2f2-42d8-b4ad-7762e5a1a437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:42:09.220287 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e27de5-a2f2-42d8-b4ad-7762e5a1a437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:09.221016 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e27de5-a2f2-42d8-b4ad-7762e5a1a437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:09.221451 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e27de5-a2f2-42d8-b4ad-7762e5a1a437 admin admin] PATH_INFO: `/v3/projects/59c77f4178e3413896315375759e337a` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:09.223464 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e27de5-a2f2-42d8-b4ad-7762e5a1a437 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59c77f4178e3413896315375759e337a)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:42:09.225785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e27de5-a2f2-42d8-b4ad-7762e5a1a437 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:42:09.279813 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-61e27de5-a2f2-42d8-b4ad-7762e5a1a437 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59c77f4178e3413896315375759e337a', 'request_id': 'req-61e27de5-a2f2-42d8-b4ad-7762e5a1a437'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:42:09.289645 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-61e27de5-a2f2-42d8-b4ad-7762e5a1a437 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59c77f4178e3413896315375759e337a', 'request_id': 'req-61e27de5-a2f2-42d8-b4ad-7762e5a1a437'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:42:09.295878 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-61e27de5-a2f2-42d8-b4ad-7762e5a1a437 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59c77f4178e3413896315375759e337a', 'request_id': 'req-61e27de5-a2f2-42d8-b4ad-7762e5a1a437'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:42:09.308344 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-61e27de5-a2f2-42d8-b4ad-7762e5a1a437 admin admin] The token cache is being invalidate because project 59c77f4178e3413896315375759e337a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:42:09.308666 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-61e27de5-a2f2-42d8-b4ad-7762e5a1a437 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:42:09.309378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-61e27de5-a2f2-42d8-b4ad-7762e5a1a437 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:42:09.314153 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1195/2404] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:42:09 2022] DELETE /identity/v3/projects/59c77f4178e3413896315375759e337a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:42:09.316689 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9a4b6-6898-4b5e-8993-18a0e481f052 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:09.317090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9a4b6-6898-4b5e-8993-18a0e481f052 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:09.317464 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9a4b6-6898-4b5e-8993-18a0e481f052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:09.331109 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b112d-844a-4cf6-9230-fc6226f93900 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:09.331429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b112d-844a-4cf6-9230-fc6226f93900 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:09.331811 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b112d-844a-4cf6-9230-fc6226f93900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:09.344472 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-a2a9a4b6-6898-4b5e-8993-18a0e481f052 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:42:09.345759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a2a9a4b6-6898-4b5e-8993-18a0e481f052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:42:09.354601 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-9d3b112d-844a-4cf6-9230-fc6226f93900 None None] MFA Rules not processed for user `40a2cbbc654749199e742ae96329a1d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:42:09.355810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-9d3b112d-844a-4cf6-9230-fc6226f93900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:42:09.366214 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:42:09.421928 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1210/2405] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:42:09.423066 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1196/2406] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:42:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 04:42:09.429772 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea400e83-fc42-4065-bb57-f96b6a9b04d3 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:09.430068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea400e83-fc42-4065-bb57-f96b6a9b04d3 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:09.430355 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea400e83-fc42-4065-bb57-f96b6a9b04d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:09.451735 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ea400e83-fc42-4065-bb57-f96b6a9b04d3 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:42:09.452942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ea400e83-fc42-4065-bb57-f96b6a9b04d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:42:09.462317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1211/2407] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:42:09.466890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b112d-844a-4cf6-9230-fc6226f93900 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:42:09.479063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b112d-844a-4cf6-9230-fc6226f93900 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:42:09.484496 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0ec3c-8e38-4427-860a-ca61b6df2477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:42:09.485118 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0ec3c-8e38-4427-860a-ca61b6df2477 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:09.485465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0ec3c-8e38-4427-860a-ca61b6df2477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:09.485809 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0ec3c-8e38-4427-860a-ca61b6df2477 admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:09.486817 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b0ec3c-8e38-4427-860a-ca61b6df2477 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:42:09.487335 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b0ec3c-8e38-4427-860a-ca61b6df2477 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:42:09.489307 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b0ec3c-8e38-4427-860a-ca61b6df2477 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:42:09.502604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1197/2408] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:42:09.507877 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea400e83-fc42-4065-bb57-f96b6a9b04d3 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:42:09.517444 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea400e83-fc42-4065-bb57-f96b6a9b04d3 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:42:09.523264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf92dc-e8f1-4de8-949c-e2be080c633a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:42:09.523889 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf92dc-e8f1-4de8-949c-e2be080c633a admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:09.524169 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf92dc-e8f1-4de8-949c-e2be080c633a admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:09.524443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf92dc-e8f1-4de8-949c-e2be080c633a admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:09.525128 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf92dc-e8f1-4de8-949c-e2be080c633a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:42:09.527086 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf92dc-e8f1-4de8-949c-e2be080c633a admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:42:09.532653 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:42:09.559013 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1212/2409] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:42:09 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:42:09.563264 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0ec3c-8e38-4427-860a-ca61b6df2477 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:42:09.573970 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0ec3c-8e38-4427-860a-ca61b6df2477 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:42:09.639953 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609aeb89-4e6e-4e88-bbb9-04181f723860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:42:09.640581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609aeb89-4e6e-4e88-bbb9-04181f723860 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:09.640889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609aeb89-4e6e-4e88-bbb9-04181f723860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:09.641164 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609aeb89-4e6e-4e88-bbb9-04181f723860 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:09.642008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609aeb89-4e6e-4e88-bbb9-04181f723860 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:42:09.643780 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609aeb89-4e6e-4e88-bbb9-04181f723860 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:42:09.670480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1198/2410] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:42:09 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 04:42:09.674528 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf92dc-e8f1-4de8-949c-e2be080c633a admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:42:09.685060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf92dc-e8f1-4de8-949c-e2be080c633a admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:42:09.689990 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50940c6b-821f-4b0f-972e-cfe6910c3954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:42:09.690611 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50940c6b-821f-4b0f-972e-cfe6910c3954 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:09.690889 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50940c6b-821f-4b0f-972e-cfe6910c3954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:09.691155 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50940c6b-821f-4b0f-972e-cfe6910c3954 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:09.692544 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50940c6b-821f-4b0f-972e-cfe6910c3954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:42:09.694368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50940c6b-821f-4b0f-972e-cfe6910c3954 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:42:09.705959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1213/2411] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:42:09.711063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609aeb89-4e6e-4e88-bbb9-04181f723860 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:42:09.722328 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609aeb89-4e6e-4e88-bbb9-04181f723860 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:42:09.727160 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0866c81-38a9-4ce4-b087-d22a0bcfb711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:42:09.727799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0866c81-38a9-4ce4-b087-d22a0bcfb711 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:09.728038 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0866c81-38a9-4ce4-b087-d22a0bcfb711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:09.728310 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0866c81-38a9-4ce4-b087-d22a0bcfb711 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:09.729008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0866c81-38a9-4ce4-b087-d22a0bcfb711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:42:09.730622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0866c81-38a9-4ce4-b087-d22a0bcfb711 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:42:09.741458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1199/2412] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 04:42:09.745677 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50940c6b-821f-4b0f-972e-cfe6910c3954 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:42:09.756791 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50940c6b-821f-4b0f-972e-cfe6910c3954 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:42:09.761629 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c13b9-010b-4445-a217-0cd9b35b3116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:42:09.762343 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c13b9-010b-4445-a217-0cd9b35b3116 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:09.762626 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c13b9-010b-4445-a217-0cd9b35b3116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:09.762936 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c13b9-010b-4445-a217-0cd9b35b3116 admin admin] PATH_INFO: `/v3/projects/cd184691f4864751a501a7c2ddaa50ea/users/edcfad2c068a49c2b26e05180a3b1b43/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:09.779289 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c13b9-010b-4445-a217-0cd9b35b3116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd184691f4864751a501a7c2ddaa50ea, user_id=edcfad2c068a49c2b26e05180a3b1b43, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:42:09.781994 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c13b9-010b-4445-a217-0cd9b35b3116 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:42:09.796056 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1214/2413] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:42:09 2022] PUT /identity/v3/projects/cd184691f4864751a501a7c2ddaa50ea/users/edcfad2c068a49c2b26e05180a3b1b43/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:42:09.807830 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b11916-1125-49e5-8c8d-a90e754cb89f None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:09.808233 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b11916-1125-49e5-8c8d-a90e754cb89f None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:09.808696 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b11916-1125-49e5-8c8d-a90e754cb89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:09.839349 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-f9b11916-1125-49e5-8c8d-a90e754cb89f None None] MFA Rules not processed for user `edcfad2c068a49c2b26e05180a3b1b43`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:42:09.840764 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-f9b11916-1125-49e5-8c8d-a90e754cb89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:42:09.913042 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1200/2414] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:42:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:42:09.925797 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c13b9-010b-4445-a217-0cd9b35b3116 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:42:09.927659 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-654c13b9-010b-4445-a217-0cd9b35b3116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:42:09.993303 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c13b9-010b-4445-a217-0cd9b35b3116 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:42:09.996977 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601f7ab-6e00-4b6b-9108-e613e1a0bc81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:42:09.997728 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601f7ab-6e00-4b6b-9108-e613e1a0bc81 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:09.997900 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601f7ab-6e00-4b6b-9108-e613e1a0bc81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:09.998164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601f7ab-6e00-4b6b-9108-e613e1a0bc81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:10.007627 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601f7ab-6e00-4b6b-9108-e613e1a0bc81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:42:10.009052 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601f7ab-6e00-4b6b-9108-e613e1a0bc81 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:42:10.017885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1215/2415] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:42:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:42:12.663374 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b11916-1125-49e5-8c8d-a90e754cb89f None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:42:12.677513 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:42:12.730481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b11916-1125-49e5-8c8d-a90e754cb89f None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:42:12.735587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe952c65-6225-402e-be90-fcb32765f1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:42:12.736270 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe952c65-6225-402e-be90-fcb32765f1a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:12.736534 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe952c65-6225-402e-be90-fcb32765f1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:12.736798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe952c65-6225-402e-be90-fcb32765f1a8 admin admin] PATH_INFO: `/v3/users/edcfad2c068a49c2b26e05180a3b1b43` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:12.738023 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe952c65-6225-402e-be90-fcb32765f1a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edcfad2c068a49c2b26e05180a3b1b43)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:42:12.740042 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe952c65-6225-402e-be90-fcb32765f1a8 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:42:12.814627 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-fe952c65-6225-402e-be90-fcb32765f1a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edcfad2c068a49c2b26e05180a3b1b43'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:42:12.815600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-fe952c65-6225-402e-be90-fcb32765f1a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edcfad2c068a49c2b26e05180a3b1b43'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:42:12.831069 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-fe952c65-6225-402e-be90-fcb32765f1a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edcfad2c068a49c2b26e05180a3b1b43'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:42:12.840533 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-fe952c65-6225-402e-be90-fcb32765f1a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edcfad2c068a49c2b26e05180a3b1b43'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:42:12.841330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-fe952c65-6225-402e-be90-fcb32765f1a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edcfad2c068a49c2b26e05180a3b1b43'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:42:12.856340 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1201/2416] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:42:12 2022] DELETE /identity/v3/users/edcfad2c068a49c2b26e05180a3b1b43 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:42:12.860918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8bea4-920e-43fb-a45f-30007ce479da None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:12.861390 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8bea4-920e-43fb-a45f-30007ce479da None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:12.861482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8bea4-920e-43fb-a45f-30007ce479da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:12.882319 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-27d8bea4-920e-43fb-a45f-30007ce479da None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:42:12.883556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-27d8bea4-920e-43fb-a45f-30007ce479da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:42:12.946944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1216/2417] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:42:12.960670 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe952c65-6225-402e-be90-fcb32765f1a8 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:42:12.962768 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-fe952c65-6225-402e-be90-fcb32765f1a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:42:13.033714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe952c65-6225-402e-be90-fcb32765f1a8 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:42:13.037977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f9b145-7acc-4625-966d-ff2eb843aae6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:42:13.038624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9b145-7acc-4625-966d-ff2eb843aae6 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:13.038892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9b145-7acc-4625-966d-ff2eb843aae6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:13.039164 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9b145-7acc-4625-966d-ff2eb843aae6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:13.049217 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f9b145-7acc-4625-966d-ff2eb843aae6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:42:13.050750 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f9b145-7acc-4625-966d-ff2eb843aae6 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:42:13.061890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1202/2418] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:42:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:42:13.490275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8bea4-920e-43fb-a45f-30007ce479da None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:42:13.491723 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-27d8bea4-920e-43fb-a45f-30007ce479da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:42:13.506076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8bea4-920e-43fb-a45f-30007ce479da None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:42:13.513593 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2869a31-ce96-4d9b-9fda-165b3c89ae3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:42:13.514452 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2869a31-ce96-4d9b-9fda-165b3c89ae3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:13.514893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2869a31-ce96-4d9b-9fda-165b3c89ae3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:13.515369 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2869a31-ce96-4d9b-9fda-165b3c89ae3f admin admin] PATH_INFO: `/v3/projects/cd184691f4864751a501a7c2ddaa50ea` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:13.516865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2869a31-ce96-4d9b-9fda-165b3c89ae3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd184691f4864751a501a7c2ddaa50ea)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:42:13.519186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2869a31-ce96-4d9b-9fda-165b3c89ae3f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:42:13.579083 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c2869a31-ce96-4d9b-9fda-165b3c89ae3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd184691f4864751a501a7c2ddaa50ea', 'request_id': 'req-c2869a31-ce96-4d9b-9fda-165b3c89ae3f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:42:13.580755 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:42:13.589968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c2869a31-ce96-4d9b-9fda-165b3c89ae3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd184691f4864751a501a7c2ddaa50ea', 'request_id': 'req-c2869a31-ce96-4d9b-9fda-165b3c89ae3f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:42:13.597141 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c2869a31-ce96-4d9b-9fda-165b3c89ae3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd184691f4864751a501a7c2ddaa50ea', 'request_id': 'req-c2869a31-ce96-4d9b-9fda-165b3c89ae3f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:42:13.610654 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c2869a31-ce96-4d9b-9fda-165b3c89ae3f admin admin] The token cache is being invalidate because project cd184691f4864751a501a7c2ddaa50ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:42:13.610950 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c2869a31-ce96-4d9b-9fda-165b3c89ae3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:42:13.611617 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c2869a31-ce96-4d9b-9fda-165b3c89ae3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:42:13.613120 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1217/2419] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:42:13 2022] DELETE /identity/v3/projects/cd184691f4864751a501a7c2ddaa50ea => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:42:13.618934 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e3086-79c5-4b9f-b686-f9f91571a96d None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:13.619221 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e3086-79c5-4b9f-b686-f9f91571a96d None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:13.619500 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e3086-79c5-4b9f-b686-f9f91571a96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:13.639655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-168e3086-79c5-4b9f-b686-f9f91571a96d None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:42:13.640879 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-168e3086-79c5-4b9f-b686-f9f91571a96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:42:13.677041 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:42:13.711579 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1203/2420] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:42:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:42:13.719470 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19500aa-3875-4a4d-a163-0ff0ea2eebd5 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:13.719756 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19500aa-3875-4a4d-a163-0ff0ea2eebd5 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:13.720033 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19500aa-3875-4a4d-a163-0ff0ea2eebd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:13.743166 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-e19500aa-3875-4a4d-a163-0ff0ea2eebd5 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:42:13.744496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e19500aa-3875-4a4d-a163-0ff0ea2eebd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:42:13.753243 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1218/2421] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:42:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:42:13.758378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e3086-79c5-4b9f-b686-f9f91571a96d None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:42:13.769833 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e3086-79c5-4b9f-b686-f9f91571a96d None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:42:13.774711 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2695b-c9de-47c6-a332-b3a0e71d0bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:42:13.775490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2695b-c9de-47c6-a332-b3a0e71d0bed admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:13.775792 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2695b-c9de-47c6-a332-b3a0e71d0bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:13.776063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2695b-c9de-47c6-a332-b3a0e71d0bed admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:13.776979 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2695b-c9de-47c6-a332-b3a0e71d0bed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 04:42:13.777284 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2695b-c9de-47c6-a332-b3a0e71d0bed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:42:13.779014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2695b-c9de-47c6-a332-b3a0e71d0bed admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:42:13.792872 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1204/2422] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 04:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 04:42:13.797677 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19500aa-3875-4a4d-a163-0ff0ea2eebd5 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:42:13.808422 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19500aa-3875-4a4d-a163-0ff0ea2eebd5 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:42:13.817337 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e8475-6bbb-45c9-aa7b-75b276457e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:42:13.817997 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e8475-6bbb-45c9-aa7b-75b276457e78 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:13.818282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e8475-6bbb-45c9-aa7b-75b276457e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:13.818557 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e8475-6bbb-45c9-aa7b-75b276457e78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:13.819245 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e8475-6bbb-45c9-aa7b-75b276457e78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:42:13.821143 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e8475-6bbb-45c9-aa7b-75b276457e78 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:42:13.851482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1219/2423] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 04:42:13 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:42:13.856483 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2695b-c9de-47c6-a332-b3a0e71d0bed admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:42:13.870915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2695b-c9de-47c6-a332-b3a0e71d0bed admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:42:13.936424 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7a322-2a50-4e77-adad-f832fc390ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:42:13.937051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7a322-2a50-4e77-adad-f832fc390ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:13.937333 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7a322-2a50-4e77-adad-f832fc390ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:13.937791 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7a322-2a50-4e77-adad-f832fc390ad2 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:13.938495 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e7a322-2a50-4e77-adad-f832fc390ad2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:42:13.940408 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e7a322-2a50-4e77-adad-f832fc390ad2 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:42:13.964088 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1205/2424] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 04:42:13 2022] POST /identity/v3/users => generated 283 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 04:42:13.968309 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e8475-6bbb-45c9-aa7b-75b276457e78 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:42:13.979426 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e8475-6bbb-45c9-aa7b-75b276457e78 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:42:13.984033 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce42394-d53b-4b6c-9cf1-0c2b82b2c45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:42:13.984642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce42394-d53b-4b6c-9cf1-0c2b82b2c45c admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:13.984948 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce42394-d53b-4b6c-9cf1-0c2b82b2c45c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:13.985213 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce42394-d53b-4b6c-9cf1-0c2b82b2c45c admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:13.986008 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce42394-d53b-4b6c-9cf1-0c2b82b2c45c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:42:13.987803 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce42394-d53b-4b6c-9cf1-0c2b82b2c45c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:42:13.999928 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1220/2425] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:42:14.005011 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7a322-2a50-4e77-adad-f832fc390ad2 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:42:14.015401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7a322-2a50-4e77-adad-f832fc390ad2 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:42:14.020421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a55f66-a011-430b-850f-b39681b10e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:42:14.021038 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a55f66-a011-430b-850f-b39681b10e1f admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:14.021321 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a55f66-a011-430b-850f-b39681b10e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:14.021655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a55f66-a011-430b-850f-b39681b10e1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:14.022366 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a55f66-a011-430b-850f-b39681b10e1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:42:14.023955 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a55f66-a011-430b-850f-b39681b10e1f admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:42:14.035386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1206/2426] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 04:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 04:42:14.039663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce42394-d53b-4b6c-9cf1-0c2b82b2c45c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:42:14.050148 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce42394-d53b-4b6c-9cf1-0c2b82b2c45c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:42:14.055911 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52efe9d6-b255-47cc-a4f5-5eca773c141f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:42:14.056588 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efe9d6-b255-47cc-a4f5-5eca773c141f admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:14.056869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efe9d6-b255-47cc-a4f5-5eca773c141f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:14.057146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efe9d6-b255-47cc-a4f5-5eca773c141f admin admin] PATH_INFO: `/v3/projects/2377e872f6fa4f3a94f24cb07f4d8e1c/users/458ff55feeae4eaba85ba47033eef700/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:14.073363 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52efe9d6-b255-47cc-a4f5-5eca773c141f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2377e872f6fa4f3a94f24cb07f4d8e1c, user_id=458ff55feeae4eaba85ba47033eef700, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:42:14.076081 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52efe9d6-b255-47cc-a4f5-5eca773c141f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:42:14.089475 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1221/2427] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 04:42:14 2022] PUT /identity/v3/projects/2377e872f6fa4f3a94f24cb07f4d8e1c/users/458ff55feeae4eaba85ba47033eef700/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:42:14.100735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636814ce-f6ee-4f71-b535-4aae177d278d None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:14.101017 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636814ce-f6ee-4f71-b535-4aae177d278d None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:14.101334 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636814ce-f6ee-4f71-b535-4aae177d278d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:14.131113 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-636814ce-f6ee-4f71-b535-4aae177d278d None None] MFA Rules not processed for user `458ff55feeae4eaba85ba47033eef700`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:42:14.132331 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-636814ce-f6ee-4f71-b535-4aae177d278d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:42:14.200448 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1207/2428] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:42:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:42:14.213088 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52efe9d6-b255-47cc-a4f5-5eca773c141f admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:42:14.215176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-52efe9d6-b255-47cc-a4f5-5eca773c141f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:42:14.283828 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52efe9d6-b255-47cc-a4f5-5eca773c141f admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:42:14.288569 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36854ae-1efc-45af-acb1-ea345c8f2038 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:42:14.289237 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36854ae-1efc-45af-acb1-ea345c8f2038 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:14.289522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36854ae-1efc-45af-acb1-ea345c8f2038 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:14.289833 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36854ae-1efc-45af-acb1-ea345c8f2038 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:14.300301 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36854ae-1efc-45af-acb1-ea345c8f2038 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:42:14.302675 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36854ae-1efc-45af-acb1-ea345c8f2038 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:42:14.312551 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1222/2429] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:42:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:42:24.197260 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=83,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:42:24.308022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=59 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:42:51.746468 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a12e3-7820-46b5-9d0d-afb81474efd4 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:51.747458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a12e3-7820-46b5-9d0d-afb81474efd4 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:51.747924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a12e3-7820-46b5-9d0d-afb81474efd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:51.774773 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-a31a12e3-7820-46b5-9d0d-afb81474efd4 None None] MFA Rules not processed for user `49a7ce9a27734dc0a187d1b217206785`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:42:51.776473 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-a31a12e3-7820-46b5-9d0d-afb81474efd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:42:51.852005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1208/2430] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:42:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:42:54.975124 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6b178-ce24-45cd-8dfe-b2e55d169ebe None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:42:54.976121 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6b178-ce24-45cd-8dfe-b2e55d169ebe None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:42:54.976646 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6b178-ce24-45cd-8dfe-b2e55d169ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:42:55.008826 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-92f6b178-ce24-45cd-8dfe-b2e55d169ebe None None] MFA Rules not processed for user `d604afd3c4234473ac1f2fcff2501743`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:42:55.010133 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-92f6b178-ce24-45cd-8dfe-b2e55d169ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:42:55.089621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1223/2431] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:42:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:43:01.841948 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=25 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:43:04.620468 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a12e3-7820-46b5-9d0d-afb81474efd4 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:43:04.689911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a12e3-7820-46b5-9d0d-afb81474efd4 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:43:04.694642 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48964b7-383e-448d-a47a-e9e90a29cfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:43:04.695326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48964b7-383e-448d-a47a-e9e90a29cfdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:43:04.695607 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48964b7-383e-448d-a47a-e9e90a29cfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:43:04.695882 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48964b7-383e-448d-a47a-e9e90a29cfdd admin admin] PATH_INFO: `/v3/users/458ff55feeae4eaba85ba47033eef700` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:43:04.697146 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48964b7-383e-448d-a47a-e9e90a29cfdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=458ff55feeae4eaba85ba47033eef700)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:43:04.699108 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48964b7-383e-448d-a47a-e9e90a29cfdd admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:43:04.749462 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c48964b7-383e-448d-a47a-e9e90a29cfdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '458ff55feeae4eaba85ba47033eef700'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:43:04.750174 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c48964b7-383e-448d-a47a-e9e90a29cfdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '458ff55feeae4eaba85ba47033eef700'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:43:04.764812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c48964b7-383e-448d-a47a-e9e90a29cfdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '458ff55feeae4eaba85ba47033eef700'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:43:04.776356 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c48964b7-383e-448d-a47a-e9e90a29cfdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '458ff55feeae4eaba85ba47033eef700'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:43:04.777053 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-c48964b7-383e-448d-a47a-e9e90a29cfdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '458ff55feeae4eaba85ba47033eef700'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:43:04.795005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1209/2432] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 04:43:04 2022] DELETE /identity/v3/users/458ff55feeae4eaba85ba47033eef700 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 04:43:04.799460 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b322cc-0991-4649-a696-adf2ce911f51 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:43:04.800017 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b322cc-0991-4649-a696-adf2ce911f51 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:43:04.800017 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b322cc-0991-4649-a696-adf2ce911f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:43:04.820042 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-89b322cc-0991-4649-a696-adf2ce911f51 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:43:04.821261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-89b322cc-0991-4649-a696-adf2ce911f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:43:04.892279 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1224/2433] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:43:04.902286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48964b7-383e-448d-a47a-e9e90a29cfdd admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:43:04.904297 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c48964b7-383e-448d-a47a-e9e90a29cfdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:43:04.965666 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48964b7-383e-448d-a47a-e9e90a29cfdd admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:43:04.969472 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004e350-743e-4331-a5c9-f46067ebc0b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:43:04.970296 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004e350-743e-4331-a5c9-f46067ebc0b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:43:04.970576 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004e350-743e-4331-a5c9-f46067ebc0b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:43:04.970845 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004e350-743e-4331-a5c9-f46067ebc0b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:43:04.980834 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004e350-743e-4331-a5c9-f46067ebc0b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:43:04.982327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004e350-743e-4331-a5c9-f46067ebc0b0 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:43:04.994064 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1210/2434] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:43:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:43:05.306850 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b322cc-0991-4649-a696-adf2ce911f51 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:43:05.308363 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-89b322cc-0991-4649-a696-adf2ce911f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:43:05.327856 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b322cc-0991-4649-a696-adf2ce911f51 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:43:05.332480 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23d796-edea-42dc-a5f5-0ce43551dcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:43:05.335604 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23d796-edea-42dc-a5f5-0ce43551dcaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:43:05.335604 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23d796-edea-42dc-a5f5-0ce43551dcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:43:05.335604 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23d796-edea-42dc-a5f5-0ce43551dcaa admin admin] PATH_INFO: `/v3/projects/2377e872f6fa4f3a94f24cb07f4d8e1c` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:43:05.337028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23d796-edea-42dc-a5f5-0ce43551dcaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=2377e872f6fa4f3a94f24cb07f4d8e1c)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:43:05.339079 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23d796-edea-42dc-a5f5-0ce43551dcaa admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:43:05.399926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6b23d796-edea-42dc-a5f5-0ce43551dcaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2377e872f6fa4f3a94f24cb07f4d8e1c', 'request_id': 'req-6b23d796-edea-42dc-a5f5-0ce43551dcaa'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:43:05.411198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6b23d796-edea-42dc-a5f5-0ce43551dcaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2377e872f6fa4f3a94f24cb07f4d8e1c', 'request_id': 'req-6b23d796-edea-42dc-a5f5-0ce43551dcaa'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:43:05.418957 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6b23d796-edea-42dc-a5f5-0ce43551dcaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2377e872f6fa4f3a94f24cb07f4d8e1c', 'request_id': 'req-6b23d796-edea-42dc-a5f5-0ce43551dcaa'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:43:05.432002 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6b23d796-edea-42dc-a5f5-0ce43551dcaa admin admin] The token cache is being invalidate because project 2377e872f6fa4f3a94f24cb07f4d8e1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 04:43:05.432432 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6b23d796-edea-42dc-a5f5-0ce43551dcaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:43:05.433200 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6b23d796-edea-42dc-a5f5-0ce43551dcaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 04:43:05.435902 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1225/2435] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 04:43:05 2022] DELETE /identity/v3/projects/2377e872f6fa4f3a94f24cb07f4d8e1c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 04:43:14.989082 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:43:15.428679 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:43:47.712327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004e350-743e-4331-a5c9-f46067ebc0b0 service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:43:47.713878 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-7004e350-743e-4331-a5c9-f46067ebc0b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:43:47.725241 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004e350-743e-4331-a5c9-f46067ebc0b0 service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:43:47.775569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19753862-5ece-4ab2-9e10-d49aab6dd2cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:43:47.776109 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19753862-5ece-4ab2-9e10-d49aab6dd2cc service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:43:47.776387 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19753862-5ece-4ab2-9e10-d49aab6dd2cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:43:47.776658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19753862-5ece-4ab2-9e10-d49aab6dd2cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:43:47.778501 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-19753862-5ece-4ab2-9e10-d49aab6dd2cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:43:47.861073 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19753862-5ece-4ab2-9e10-d49aab6dd2cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:43:47.862541 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19753862-5ece-4ab2-9e10-d49aab6dd2cc service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:43:47.874037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1211/2436] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:43:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:43:49.057419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23d796-edea-42dc-a5f5-0ce43551dcaa admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:43:49.058928 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6b23d796-edea-42dc-a5f5-0ce43551dcaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:43:49.071346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23d796-edea-42dc-a5f5-0ce43551dcaa admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:43:49.074861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1cd06-14e7-4cdd-846d-2483c63bb341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:43:49.075468 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1cd06-14e7-4cdd-846d-2483c63bb341 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:43:49.075746 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1cd06-14e7-4cdd-846d-2483c63bb341 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:43:49.076014 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1cd06-14e7-4cdd-846d-2483c63bb341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:43:49.077982 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-13d1cd06-14e7-4cdd-846d-2483c63bb341 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:43:49.091666 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1cd06-14e7-4cdd-846d-2483c63bb341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:43:49.093811 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1cd06-14e7-4cdd-846d-2483c63bb341 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:43:49.105072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1226/2437] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:43:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:43:57.868993 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=54 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:43:59.099946 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=6 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:44:04.621214 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19753862-5ece-4ab2-9e10-d49aab6dd2cc service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:44:04.623817 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-19753862-5ece-4ab2-9e10-d49aab6dd2cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:44:04.699644 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19753862-5ece-4ab2-9e10-d49aab6dd2cc service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:44:04.704188 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fcee49-fc2a-4be3-b316-88c9188c01eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:44:04.704830 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fcee49-fc2a-4be3-b316-88c9188c01eb service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:44:04.705113 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fcee49-fc2a-4be3-b316-88c9188c01eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:44:04.705388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fcee49-fc2a-4be3-b316-88c9188c01eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:44:04.707806 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-41fcee49-fc2a-4be3-b316-88c9188c01eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:44:04.787801 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fcee49-fc2a-4be3-b316-88c9188c01eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:44:04.789579 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fcee49-fc2a-4be3-b316-88c9188c01eb service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:44:04.802586 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1212/2438] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:44:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:44:08.168910 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1cd06-14e7-4cdd-846d-2483c63bb341 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:44:08.171032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-13d1cd06-14e7-4cdd-846d-2483c63bb341 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:44:08.186110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1cd06-14e7-4cdd-846d-2483c63bb341 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:44:08.191713 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c5bbd-e514-44b2-a267-0060a884ed2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:44:08.192328 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c5bbd-e514-44b2-a267-0060a884ed2c service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:44:08.192611 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c5bbd-e514-44b2-a267-0060a884ed2c service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:44:08.192886 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c5bbd-e514-44b2-a267-0060a884ed2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:44:08.194672 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-210c5bbd-e514-44b2-a267-0060a884ed2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:44:08.269055 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c5bbd-e514-44b2-a267-0060a884ed2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:44:08.272444 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c5bbd-e514-44b2-a267-0060a884ed2c service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:44:08.282750 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1227/2439] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:44:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:44:13.268412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fcee49-fc2a-4be3-b316-88c9188c01eb service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:44:13.275188 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-41fcee49-fc2a-4be3-b316-88c9188c01eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:44:13.297440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fcee49-fc2a-4be3-b316-88c9188c01eb service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:44:13.308465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e2a63d-b641-4550-b4ce-733630c337d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:44:13.309121 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2a63d-b641-4550-b4ce-733630c337d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:44:13.309446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2a63d-b641-4550-b4ce-733630c337d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:44:13.310336 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2a63d-b641-4550-b4ce-733630c337d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:44:13.316016 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-77e2a63d-b641-4550-b4ce-733630c337d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:44:13.338586 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e2a63d-b641-4550-b4ce-733630c337d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:44:13.340323 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e2a63d-b641-4550-b4ce-733630c337d8 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:44:13.350191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1213/2440] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:44:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:44:16.582959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73283b-b730-4c5b-9ca1-040d3c68f7b9 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:44:16.583494 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73283b-b730-4c5b-9ca1-040d3c68f7b9 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:44:16.583606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73283b-b730-4c5b-9ca1-040d3c68f7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:44:16.603589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-fa73283b-b730-4c5b-9ca1-040d3c68f7b9 None None] MFA Rules not processed for user `12f7b1c19d824ac7b8207bf001b79057`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:44:16.604796 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fa73283b-b730-4c5b-9ca1-040d3c68f7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:44:16.619008 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1228/2441] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:44:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:44:16.994732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e2a63d-b641-4550-b4ce-733630c337d8 service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:44:17.004500 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e2a63d-b641-4550-b4ce-733630c337d8 service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:44:17.008306 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70063f-8c0d-490d-83d0-6d85e2ec6d28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:44:17.008958 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70063f-8c0d-490d-83d0-6d85e2ec6d28 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:44:17.009238 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70063f-8c0d-490d-83d0-6d85e2ec6d28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:44:17.009512 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70063f-8c0d-490d-83d0-6d85e2ec6d28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:44:17.011348 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-2e70063f-8c0d-490d-83d0-6d85e2ec6d28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:44:17.024723 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e70063f-8c0d-490d-83d0-6d85e2ec6d28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:44:17.026207 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e70063f-8c0d-490d-83d0-6d85e2ec6d28 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:44:17.035685 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1214/2442] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:44:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:44:17.378031 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa73283b-b730-4c5b-9ca1-040d3c68f7b9 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:44:17.379463 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fa73283b-b730-4c5b-9ca1-040d3c68f7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:44:17.462910 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa73283b-b730-4c5b-9ca1-040d3c68f7b9 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:44:17.469784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a220a-89cd-465e-94d6-431fc767379a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350f72f740f749f68175c1c4e8ae9d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:44:17.470413 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a220a-89cd-465e-94d6-431fc767379a service placement] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:44:17.470692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a220a-89cd-465e-94d6-431fc767379a service placement] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:44:17.470965 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a220a-89cd-465e-94d6-431fc767379a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:44:17.472760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f58a220a-89cd-465e-94d6-431fc767379a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:44:17.485232 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58a220a-89cd-465e-94d6-431fc767379a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:44:17.486774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58a220a-89cd-465e-94d6-431fc767379a service placement] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:44:17.499079 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1229/2443] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:44:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:44:27.031738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=60 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:44:27.493002 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=63 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:44:28.098416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70063f-8c0d-490d-83d0-6d85e2ec6d28 service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:44:28.112980 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70063f-8c0d-490d-83d0-6d85e2ec6d28 service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:44:28.119602 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700bebc-b125-402c-ab46-aa2be0cd7f75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:44:28.120245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700bebc-b125-402c-ab46-aa2be0cd7f75 service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:44:28.120531 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700bebc-b125-402c-ab46-aa2be0cd7f75 service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:44:28.120805 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700bebc-b125-402c-ab46-aa2be0cd7f75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:44:28.123361 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-3700bebc-b125-402c-ab46-aa2be0cd7f75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:44:28.136285 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700bebc-b125-402c-ab46-aa2be0cd7f75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:44:28.138514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700bebc-b125-402c-ab46-aa2be0cd7f75 service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:44:28.150550 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1215/2444] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:44:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:44:33.874749 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48927a-d0c5-4853-bfce-e60b2b376fcc None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:44:33.875352 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48927a-d0c5-4853-bfce-e60b2b376fcc None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:44:33.875352 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48927a-d0c5-4853-bfce-e60b2b376fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:44:33.895019 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-2f48927a-d0c5-4853-bfce-e60b2b376fcc None None] MFA Rules not processed for user `40a2cbbc654749199e742ae96329a1d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:44:33.896276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2f48927a-d0c5-4853-bfce-e60b2b376fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:44:33.958154 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1230/2445] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:44:33 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:44:38.145270 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=6 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:44:43.951328 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=25 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:45:14.789539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700bebc-b125-402c-ab46-aa2be0cd7f75 service nova] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:45:14.802117 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700bebc-b125-402c-ab46-aa2be0cd7f75 service nova] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:45:14.809017 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93daa233-277f-48fa-ac33-c02fa14c0553 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:45:14.809685 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93daa233-277f-48fa-ac33-c02fa14c0553 service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:45:14.809964 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93daa233-277f-48fa-ac33-c02fa14c0553 service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:45:14.810233 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93daa233-277f-48fa-ac33-c02fa14c0553 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:45:14.812026 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-93daa233-277f-48fa-ac33-c02fa14c0553 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:45:14.822842 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93daa233-277f-48fa-ac33-c02fa14c0553 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:45:14.824404 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93daa233-277f-48fa-ac33-c02fa14c0553 service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:45:14.833697 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1216/2446] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:45:14 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:45:24.829350 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=6 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:45:53.668155 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48927a-d0c5-4853-bfce-e60b2b376fcc None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:45:53.669536 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2f48927a-d0c5-4853-bfce-e60b2b376fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:45:53.686364 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48927a-d0c5-4853-bfce-e60b2b376fcc None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:45:53.691772 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6123b-ea59-4da7-893d-38db9cac3403 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:45:53.692379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6123b-ea59-4da7-893d-38db9cac3403 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:45:53.692662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6123b-ea59-4da7-893d-38db9cac3403 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:45:53.692936 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6123b-ea59-4da7-893d-38db9cac3403 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:45:53.695732 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-46a6123b-ea59-4da7-893d-38db9cac3403 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:45:53.753024 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6123b-ea59-4da7-893d-38db9cac3403 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:45:53.754698 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6123b-ea59-4da7-893d-38db9cac3403 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:45:53.763571 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1231/2447] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:45:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:46:03.759637 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=28 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:46:35.213485 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93daa233-277f-48fa-ac33-c02fa14c0553 service nova] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:46:35.223608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93daa233-277f-48fa-ac33-c02fa14c0553 service nova] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:46:35.227350 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77866f-bbc2-4933-b262-d8b7493c5e6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:46:35.228233 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77866f-bbc2-4933-b262-d8b7493c5e6e service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:46:35.228233 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77866f-bbc2-4933-b262-d8b7493c5e6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:46:35.228515 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77866f-bbc2-4933-b262-d8b7493c5e6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:46:35.231219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-8a77866f-bbc2-4933-b262-d8b7493c5e6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:46:35.296692 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77866f-bbc2-4933-b262-d8b7493c5e6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:46:35.298179 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77866f-bbc2-4933-b262-d8b7493c5e6e service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:46:35.308416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1217/2448] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:46:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:46:45.303899 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=28 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:47:03.665701 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6123b-ea59-4da7-893d-38db9cac3403 service nova] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:47:03.730523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6123b-ea59-4da7-893d-38db9cac3403 service nova] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:47:03.735618 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed5d91-a994-4a1c-b0b0-558b4dc2398b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350f72f740f749f68175c1c4e8ae9d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:47:03.736227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed5d91-a994-4a1c-b0b0-558b4dc2398b service placement] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:47:03.736509 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed5d91-a994-4a1c-b0b0-558b4dc2398b service placement] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:47:03.736782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed5d91-a994-4a1c-b0b0-558b4dc2398b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:47:03.739159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-18ed5d91-a994-4a1c-b0b0-558b4dc2398b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:47:03.771380 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed5d91-a994-4a1c-b0b0-558b4dc2398b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:47:03.772981 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed5d91-a994-4a1c-b0b0-558b4dc2398b service placement] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:47:03.782994 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1232/2449] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:47:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:47:13.778873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=36 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:49:04.635742 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77866f-bbc2-4933-b262-d8b7493c5e6e service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:49:04.637190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-8a77866f-bbc2-4933-b262-d8b7493c5e6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:49:04.692578 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77866f-bbc2-4933-b262-d8b7493c5e6e service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:49:04.696177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5ab29-e430-4765-b99b-bedac009937c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:49:04.696789 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5ab29-e430-4765-b99b-bedac009937c service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:49:04.697063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5ab29-e430-4765-b99b-bedac009937c service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:49:04.697354 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5ab29-e430-4765-b99b-bedac009937c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:49:04.705877 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf5ab29-e430-4765-b99b-bedac009937c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:49:04.707274 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf5ab29-e430-4765-b99b-bedac009937c service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:49:04.733882 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1218/2450] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:49:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:49:09.021232 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed5d91-a994-4a1c-b0b0-558b4dc2398b service placement] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:49:09.030398 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed5d91-a994-4a1c-b0b0-558b4dc2398b service placement] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:49:09.034509 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703839e8-93dd-4d29-bca3-e0ba55a6199e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:49:09.035104 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703839e8-93dd-4d29-bca3-e0ba55a6199e service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:49:09.035379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703839e8-93dd-4d29-bca3-e0ba55a6199e service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:49:09.035646 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703839e8-93dd-4d29-bca3-e0ba55a6199e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:49:09.044128 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703839e8-93dd-4d29-bca3-e0ba55a6199e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:49:09.045459 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703839e8-93dd-4d29-bca3-e0ba55a6199e service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:49:09.071884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1233/2451] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:49:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:49:14.731345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=31 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:49:19.069365 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=13 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:49:21.735774 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5ab29-e430-4765-b99b-bedac009937c service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:49:21.744573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5ab29-e430-4765-b99b-bedac009937c service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:49:21.748278 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23884ece-c637-4a47-90ad-c63e501fb886 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:49:21.748882 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23884ece-c637-4a47-90ad-c63e501fb886 service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:49:21.749163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23884ece-c637-4a47-90ad-c63e501fb886 service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:49:21.749437 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23884ece-c637-4a47-90ad-c63e501fb886 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:49:21.767015 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23884ece-c637-4a47-90ad-c63e501fb886 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:49:21.769101 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23884ece-c637-4a47-90ad-c63e501fb886 service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:49:21.790068 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1219/2452] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:49:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:49:31.787254 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=13 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:49:51.283359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703839e8-93dd-4d29-bca3-e0ba55a6199e service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:49:51.303846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703839e8-93dd-4d29-bca3-e0ba55a6199e service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:49:51.307384 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ef7d5-b196-4aca-9206-cc1781244dd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:49:51.307987 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ef7d5-b196-4aca-9206-cc1781244dd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:49:51.308268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ef7d5-b196-4aca-9206-cc1781244dd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:49:51.308539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ef7d5-b196-4aca-9206-cc1781244dd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:49:51.319379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ef7d5-b196-4aca-9206-cc1781244dd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:49:51.320797 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ef7d5-b196-4aca-9206-cc1781244dd5 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:49:51.332388 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1234/2453] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:49:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:49:52.475867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23884ece-c637-4a47-90ad-c63e501fb886 service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:49:52.489512 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23884ece-c637-4a47-90ad-c63e501fb886 service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:49:52.497453 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b60b33-6629-4172-8803-ebeab0fd04d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:49:52.498260 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b60b33-6629-4172-8803-ebeab0fd04d9 service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:49:52.498618 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b60b33-6629-4172-8803-ebeab0fd04d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:49:52.499019 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b60b33-6629-4172-8803-ebeab0fd04d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:49:52.501160 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c5b60b33-6629-4172-8803-ebeab0fd04d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:49:52.552822 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b60b33-6629-4172-8803-ebeab0fd04d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:49:52.555297 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b60b33-6629-4172-8803-ebeab0fd04d9 service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:49:52.583812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1220/2454] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:49:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:49:55.803337 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db0137-7ac5-4686-bb2b-c6a4e4608486 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:49:55.803867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db0137-7ac5-4686-bb2b-c6a4e4608486 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:49:55.803867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db0137-7ac5-4686-bb2b-c6a4e4608486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:49:55.843415 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-68db0137-7ac5-4686-bb2b-c6a4e4608486 None None] MFA Rules not processed for user `49a7ce9a27734dc0a187d1b217206785`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 04:49:55.844593 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-68db0137-7ac5-4686-bb2b-c6a4e4608486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:49:55.851686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1235/2455] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 04:49:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 04:49:56.163077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b60b33-6629-4172-8803-ebeab0fd04d9 service nova] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:49:56.172519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b60b33-6629-4172-8803-ebeab0fd04d9 service nova] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:49:56.177114 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396c286-7460-4f4f-a1c1-06a90f59e5f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:49:56.177758 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396c286-7460-4f4f-a1c1-06a90f59e5f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:49:56.178039 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396c286-7460-4f4f-a1c1-06a90f59e5f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:49:56.178308 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396c286-7460-4f4f-a1c1-06a90f59e5f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:49:56.185500 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396c286-7460-4f4f-a1c1-06a90f59e5f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:49:56.187021 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396c286-7460-4f4f-a1c1-06a90f59e5f7 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:49:56.194857 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1221/2456] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:49:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:49:56.529789 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db0137-7ac5-4686-bb2b-c6a4e4608486 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:49:56.546024 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db0137-7ac5-4686-bb2b-c6a4e4608486 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:49:56.551895 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59a7ef-886f-4491-8fb3-be7767049098 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350f72f740f749f68175c1c4e8ae9d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:49:56.552659 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59a7ef-886f-4491-8fb3-be7767049098 service placement] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:49:56.552943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59a7ef-886f-4491-8fb3-be7767049098 service placement] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:49:56.553216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59a7ef-886f-4491-8fb3-be7767049098 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:49:56.571671 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59a7ef-886f-4491-8fb3-be7767049098 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:49:56.573967 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59a7ef-886f-4491-8fb3-be7767049098 service placement] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:49:56.594001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1236/2457] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:49:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:49:58.236699 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396c286-7460-4f4f-a1c1-06a90f59e5f7 service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:49:58.246266 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396c286-7460-4f4f-a1c1-06a90f59e5f7 service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:49:58.250173 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664ff4d-c79a-4177-8fd6-5ffe8c447263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:49:58.250790 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664ff4d-c79a-4177-8fd6-5ffe8c447263 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:49:58.251073 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664ff4d-c79a-4177-8fd6-5ffe8c447263 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:49:58.251349 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664ff4d-c79a-4177-8fd6-5ffe8c447263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:49:58.259816 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b664ff4d-c79a-4177-8fd6-5ffe8c447263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:49:58.261257 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b664ff4d-c79a-4177-8fd6-5ffe8c447263 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:49:58.270196 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1222/2458] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:49:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:50:03.634816 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59a7ef-886f-4491-8fb3-be7767049098 service placement] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:50:03.648850 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59a7ef-886f-4491-8fb3-be7767049098 service placement] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:50:03.655604 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac4a7d-97dd-42db-9f42-646b7580958e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:50:03.656214 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac4a7d-97dd-42db-9f42-646b7580958e service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:50:03.656490 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac4a7d-97dd-42db-9f42-646b7580958e service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:50:03.656761 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac4a7d-97dd-42db-9f42-646b7580958e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:50:03.658543 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-44ac4a7d-97dd-42db-9f42-646b7580958e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:50:03.672592 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac4a7d-97dd-42db-9f42-646b7580958e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:50:03.674339 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac4a7d-97dd-42db-9f42-646b7580958e service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:50:03.686698 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1237/2459] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:50:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:50:06.977826 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664ff4d-c79a-4177-8fd6-5ffe8c447263 service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:50:06.992006 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664ff4d-c79a-4177-8fd6-5ffe8c447263 service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:50:06.998987 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d2c20-daf6-49e0-9a81-710051a5b063 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:50:06.999651 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d2c20-daf6-49e0-9a81-710051a5b063 service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:50:07.000115 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d2c20-daf6-49e0-9a81-710051a5b063 service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:50:07.000385 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d2c20-daf6-49e0-9a81-710051a5b063 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:50:07.011408 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d2c20-daf6-49e0-9a81-710051a5b063 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:50:07.013051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d2c20-daf6-49e0-9a81-710051a5b063 service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:50:07.025285 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1223/2460] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:50:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:50:13.681825 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=33 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:50:14.403216 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac4a7d-97dd-42db-9f42-646b7580958e service nova] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:50:14.414694 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac4a7d-97dd-42db-9f42-646b7580958e service nova] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:50:14.420312 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ad2d0-dc48-44b7-a634-36888d87eb92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:50:14.420911 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ad2d0-dc48-44b7-a634-36888d87eb92 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:50:14.421188 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ad2d0-dc48-44b7-a634-36888d87eb92 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:50:14.421456 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ad2d0-dc48-44b7-a634-36888d87eb92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:50:14.429749 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ad2d0-dc48-44b7-a634-36888d87eb92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:50:14.431361 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ad2d0-dc48-44b7-a634-36888d87eb92 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:50:14.460082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1238/2461] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:50:14 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:50:17.021315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=29 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:50:24.457710 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=13 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:50:54.781317 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d2c20-daf6-49e0-9a81-710051a5b063 service nova] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:50:54.791326 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d2c20-daf6-49e0-9a81-710051a5b063 service nova] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:50:54.795133 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7455b-1c5a-40d7-9c08-ea39aaf26f3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:50:54.795726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7455b-1c5a-40d7-9c08-ea39aaf26f3f service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:50:54.796007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7455b-1c5a-40d7-9c08-ea39aaf26f3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:50:54.796280 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7455b-1c5a-40d7-9c08-ea39aaf26f3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:50:54.804910 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7455b-1c5a-40d7-9c08-ea39aaf26f3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:50:54.806268 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7455b-1c5a-40d7-9c08-ea39aaf26f3f service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:50:54.833039 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1224/2462] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:50:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:51:04.829981 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=13 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:51:35.198244 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ad2d0-dc48-44b7-a634-36888d87eb92 service nova] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:51:35.207802 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ad2d0-dc48-44b7-a634-36888d87eb92 service nova] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:51:35.212003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4f448-67c7-4ecd-98e5-48a8d0f31c86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:51:35.212607 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4f448-67c7-4ecd-98e5-48a8d0f31c86 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:51:35.212952 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4f448-67c7-4ecd-98e5-48a8d0f31c86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:51:35.213284 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4f448-67c7-4ecd-98e5-48a8d0f31c86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:51:35.223475 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4f448-67c7-4ecd-98e5-48a8d0f31c86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:51:35.226374 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4f448-67c7-4ecd-98e5-48a8d0f31c86 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:51:35.237237 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1239/2463] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:51:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:51:45.231990 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=6 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:52:10.650851 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7455b-1c5a-40d7-9c08-ea39aaf26f3f service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:52:10.652432 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-38c7455b-1c5a-40d7-9c08-ea39aaf26f3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:52:10.678002 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7455b-1c5a-40d7-9c08-ea39aaf26f3f service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:52:10.683080 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e754a8e1-31fa-4a16-a1ca-7a109846c6b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350f72f740f749f68175c1c4e8ae9d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:52:10.683696 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754a8e1-31fa-4a16-a1ca-7a109846c6b9 service placement] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:52:10.684076 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754a8e1-31fa-4a16-a1ca-7a109846c6b9 service placement] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:52:10.684924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754a8e1-31fa-4a16-a1ca-7a109846c6b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:52:10.694063 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e754a8e1-31fa-4a16-a1ca-7a109846c6b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:52:10.695642 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e754a8e1-31fa-4a16-a1ca-7a109846c6b9 service placement] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:52:10.706067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1225/2464] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:52:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:52:20.701604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=6 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:54:04.813728 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4f448-67c7-4ecd-98e5-48a8d0f31c86 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:54:04.866576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4f448-67c7-4ecd-98e5-48a8d0f31c86 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:54:04.870225 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51acc7ab-0ba2-4719-a895-eb75af4cf953 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:54:04.870837 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51acc7ab-0ba2-4719-a895-eb75af4cf953 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:54:04.871115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51acc7ab-0ba2-4719-a895-eb75af4cf953 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:54:04.871386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51acc7ab-0ba2-4719-a895-eb75af4cf953 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:54:04.873257 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-51acc7ab-0ba2-4719-a895-eb75af4cf953 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:54:04.931247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51acc7ab-0ba2-4719-a895-eb75af4cf953 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:54:04.932595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51acc7ab-0ba2-4719-a895-eb75af4cf953 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:54:04.941880 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1240/2465] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:54:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:54:09.596041 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e754a8e1-31fa-4a16-a1ca-7a109846c6b9 service placement] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:54:09.605484 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e754a8e1-31fa-4a16-a1ca-7a109846c6b9 service placement] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:54:09.609810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00ebb6-31ab-4b60-934c-115274b4ae32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:54:09.610591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00ebb6-31ab-4b60-934c-115274b4ae32 service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:54:09.610873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00ebb6-31ab-4b60-934c-115274b4ae32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:54:09.611148 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00ebb6-31ab-4b60-934c-115274b4ae32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:54:09.613047 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-be00ebb6-31ab-4b60-934c-115274b4ae32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:54:09.684657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be00ebb6-31ab-4b60-934c-115274b4ae32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:54:09.686152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be00ebb6-31ab-4b60-934c-115274b4ae32 service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:54:09.695073 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1226/2466] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:54:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:54:14.937291 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=48 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:54:19.690885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=33 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:55:06.373794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51acc7ab-0ba2-4719-a895-eb75af4cf953 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:55:06.375352 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-51acc7ab-0ba2-4719-a895-eb75af4cf953 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:55:06.441590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51acc7ab-0ba2-4719-a895-eb75af4cf953 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:55:06.445811 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9c2fc-005e-483c-95b0-74de044ec36d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:55:06.446418 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9c2fc-005e-483c-95b0-74de044ec36d service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:55:06.446695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9c2fc-005e-483c-95b0-74de044ec36d service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:55:06.446966 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9c2fc-005e-483c-95b0-74de044ec36d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:55:06.448816 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d8a9c2fc-005e-483c-95b0-74de044ec36d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:55:06.457999 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9c2fc-005e-483c-95b0-74de044ec36d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:55:06.460133 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9c2fc-005e-483c-95b0-74de044ec36d service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:55:06.468154 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1241/2467] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:55:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:55:13.508541 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00ebb6-31ab-4b60-934c-115274b4ae32 service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:55:13.510201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-be00ebb6-31ab-4b60-934c-115274b4ae32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:55:13.520898 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00ebb6-31ab-4b60-934c-115274b4ae32 service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:55:13.524952 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99dbebd-e89c-4356-b7e5-c1cdf241df2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:55:13.525637 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dbebd-e89c-4356-b7e5-c1cdf241df2c service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:55:13.525930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dbebd-e89c-4356-b7e5-c1cdf241df2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:55:13.526206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dbebd-e89c-4356-b7e5-c1cdf241df2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:55:13.583003 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99dbebd-e89c-4356-b7e5-c1cdf241df2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:55:13.584502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99dbebd-e89c-4356-b7e5-c1cdf241df2c service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:55:13.593443 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1227/2468] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:55:13 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:55:16.464683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=31 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:55:21.678397 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9c2fc-005e-483c-95b0-74de044ec36d service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:55:21.680180 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d8a9c2fc-005e-483c-95b0-74de044ec36d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:55:21.690404 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9c2fc-005e-483c-95b0-74de044ec36d service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:55:21.694826 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae362b8-6712-4c5c-92ff-29c8843ea627 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:55:21.695426 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae362b8-6712-4c5c-92ff-29c8843ea627 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:55:21.695762 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae362b8-6712-4c5c-92ff-29c8843ea627 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:55:21.696027 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae362b8-6712-4c5c-92ff-29c8843ea627 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:55:21.697903 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4ae362b8-6712-4c5c-92ff-29c8843ea627 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:55:21.768085 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae362b8-6712-4c5c-92ff-29c8843ea627 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:55:21.769508 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae362b8-6712-4c5c-92ff-29c8843ea627 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:55:21.780474 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1242/2469] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:55:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:55:22.959893 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99dbebd-e89c-4356-b7e5-c1cdf241df2c service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:55:22.969110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99dbebd-e89c-4356-b7e5-c1cdf241df2c service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:55:22.972728 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15924d-e4da-40a4-a8c0-3914af61569d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:55:22.973323 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15924d-e4da-40a4-a8c0-3914af61569d service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:55:22.973636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15924d-e4da-40a4-a8c0-3914af61569d service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:55:22.973908 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15924d-e4da-40a4-a8c0-3914af61569d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:55:22.976222 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-1e15924d-e4da-40a4-a8c0-3914af61569d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:55:22.989212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15924d-e4da-40a4-a8c0-3914af61569d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:55:22.990894 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15924d-e4da-40a4-a8c0-3914af61569d service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:55:23.002174 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1228/2470] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:55:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:55:31.775769 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=34 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:55:32.997070 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=34 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:55:54.052006 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae362b8-6712-4c5c-92ff-29c8843ea627 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:55:54.062049 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae362b8-6712-4c5c-92ff-29c8843ea627 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:55:54.065530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a8ace-4a9e-4c4f-8852-05b3a14c6254 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:55:54.066191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a8ace-4a9e-4c4f-8852-05b3a14c6254 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:55:54.066468 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a8ace-4a9e-4c4f-8852-05b3a14c6254 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:55:54.066740 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a8ace-4a9e-4c4f-8852-05b3a14c6254 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:55:54.068568 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-995a8ace-4a9e-4c4f-8852-05b3a14c6254 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:55:54.137242 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a8ace-4a9e-4c4f-8852-05b3a14c6254 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:55:54.138700 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a8ace-4a9e-4c4f-8852-05b3a14c6254 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:55:54.147397 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1243/2471] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:55:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:55:55.489592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15924d-e4da-40a4-a8c0-3914af61569d service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:55:55.491306 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-1e15924d-e4da-40a4-a8c0-3914af61569d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:55:55.506289 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15924d-e4da-40a4-a8c0-3914af61569d service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:55:55.517131 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a03d73-ea79-4cff-b264-3ec5bcb7d8ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:55:55.517884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a03d73-ea79-4cff-b264-3ec5bcb7d8ad service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:55:55.518167 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a03d73-ea79-4cff-b264-3ec5bcb7d8ad service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:55:55.518573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a03d73-ea79-4cff-b264-3ec5bcb7d8ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:55:55.528886 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a03d73-ea79-4cff-b264-3ec5bcb7d8ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:55:55.530494 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a03d73-ea79-4cff-b264-3ec5bcb7d8ad service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:55:55.543411 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1229/2472] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:55:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 04:56:04.143405 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=33 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:56:05.538366 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=6 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:56:35.795825 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a8ace-4a9e-4c4f-8852-05b3a14c6254 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:56:35.809233 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a8ace-4a9e-4c4f-8852-05b3a14c6254 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:56:35.814809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b46dd8-8a93-4275-a161-466bbbb4b0f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:56:35.815414 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b46dd8-8a93-4275-a161-466bbbb4b0f1 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:56:35.815697 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b46dd8-8a93-4275-a161-466bbbb4b0f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:56:35.815973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b46dd8-8a93-4275-a161-466bbbb4b0f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:56:35.817884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-86b46dd8-8a93-4275-a161-466bbbb4b0f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:56:35.898359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b46dd8-8a93-4275-a161-466bbbb4b0f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:56:35.900115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b46dd8-8a93-4275-a161-466bbbb4b0f1 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:56:35.918637 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1244/2473] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:56:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:56:45.914988 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=41 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:57:16.698338 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a03d73-ea79-4cff-b264-3ec5bcb7d8ad service nova] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:57:16.700519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-65a03d73-ea79-4cff-b264-3ec5bcb7d8ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:57:16.945025 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a03d73-ea79-4cff-b264-3ec5bcb7d8ad service nova] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:57:16.951010 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7709d-b74b-460c-8707-edca181f8ae8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350f72f740f749f68175c1c4e8ae9d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:57:16.951632 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7709d-b74b-460c-8707-edca181f8ae8 service placement] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:57:16.951919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7709d-b74b-460c-8707-edca181f8ae8 service placement] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:57:16.952321 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7709d-b74b-460c-8707-edca181f8ae8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:57:16.954452 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-68f7709d-b74b-460c-8707-edca181f8ae8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:57:16.967456 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7709d-b74b-460c-8707-edca181f8ae8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:57:16.969052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7709d-b74b-460c-8707-edca181f8ae8 service placement] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:57:16.979244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1230/2474] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:57:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:57:26.974975 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=54 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:59:55.826168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b46dd8-8a93-4275-a161-466bbbb4b0f1 service nova] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:59:55.859609 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b46dd8-8a93-4275-a161-466bbbb4b0f1 service nova] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:59:55.864049 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a99c6-d700-4a75-b852-5bef9af7fb03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:59:55.864769 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a99c6-d700-4a75-b852-5bef9af7fb03 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:59:55.865116 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a99c6-d700-4a75-b852-5bef9af7fb03 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:59:55.865397 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a99c6-d700-4a75-b852-5bef9af7fb03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:59:55.875120 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a99c6-d700-4a75-b852-5bef9af7fb03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:59:55.876991 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a99c6-d700-4a75-b852-5bef9af7fb03 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:59:55.906225 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1245/2475] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:59:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:59:55.950268 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7709d-b74b-460c-8707-edca181f8ae8 service placement] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:59:55.970204 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7709d-b74b-460c-8707-edca181f8ae8 service placement] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:59:55.979793 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed8e3ce-743a-468d-a6ca-d683e848911a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:59:55.980421 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8e3ce-743a-468d-a6ca-d683e848911a service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:59:55.980699 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8e3ce-743a-468d-a6ca-d683e848911a service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:59:55.980967 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8e3ce-743a-468d-a6ca-d683e848911a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:59:55.989924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed8e3ce-743a-468d-a6ca-d683e848911a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:59:55.991334 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed8e3ce-743a-468d-a6ca-d683e848911a service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:59:56.020967 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1231/2476] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 04:59:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:00:05.903443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=22 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:00:06.018281 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=18 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:00:12.395898 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a99c6-d700-4a75-b852-5bef9af7fb03 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:00:12.415375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a99c6-d700-4a75-b852-5bef9af7fb03 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:00:12.419010 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2dd662-3cf8-4f01-8d3a-7e9e5917f8e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:00:12.419617 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2dd662-3cf8-4f01-8d3a-7e9e5917f8e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:00:12.419899 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2dd662-3cf8-4f01-8d3a-7e9e5917f8e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:00:12.420180 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2dd662-3cf8-4f01-8d3a-7e9e5917f8e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:00:12.428476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2dd662-3cf8-4f01-8d3a-7e9e5917f8e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:00:12.429827 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2dd662-3cf8-4f01-8d3a-7e9e5917f8e1 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:00:12.438475 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1246/2477] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:00:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:00:13.672601 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed8e3ce-743a-468d-a6ca-d683e848911a service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:00:13.702822 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed8e3ce-743a-468d-a6ca-d683e848911a service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:00:13.708569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b99c2-deab-4de0-b8d7-231bc50ff1ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:00:13.709187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b99c2-deab-4de0-b8d7-231bc50ff1ea service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:00:13.709466 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b99c2-deab-4de0-b8d7-231bc50ff1ea service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:00:13.709770 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b99c2-deab-4de0-b8d7-231bc50ff1ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:00:13.711652 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-7a6b99c2-deab-4de0-b8d7-231bc50ff1ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:00:13.747196 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b99c2-deab-4de0-b8d7-231bc50ff1ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:00:13.748683 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b99c2-deab-4de0-b8d7-231bc50ff1ea service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:00:13.757884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1232/2478] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:00:13 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:00:22.099204 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2dd662-3cf8-4f01-8d3a-7e9e5917f8e1 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:00:22.107589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2dd662-3cf8-4f01-8d3a-7e9e5917f8e1 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:00:22.111783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7faa9-a7a4-44b9-835e-6fd1b0ffa525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:00:22.113057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7faa9-a7a4-44b9-835e-6fd1b0ffa525 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:00:22.113340 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7faa9-a7a4-44b9-835e-6fd1b0ffa525 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:00:22.113650 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7faa9-a7a4-44b9-835e-6fd1b0ffa525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:00:22.124645 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac7faa9-a7a4-44b9-835e-6fd1b0ffa525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:00:22.126134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac7faa9-a7a4-44b9-835e-6fd1b0ffa525 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:00:22.138534 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1247/2479] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:00:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:00:22.776668 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b99c2-deab-4de0-b8d7-231bc50ff1ea service nova] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:00:22.785846 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b99c2-deab-4de0-b8d7-231bc50ff1ea service nova] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:00:22.789375 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6625b4c5-c4cb-4d9a-a4f1-d39042c54e89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:00:22.790008 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625b4c5-c4cb-4d9a-a4f1-d39042c54e89 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:00:22.790287 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625b4c5-c4cb-4d9a-a4f1-d39042c54e89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:00:22.790552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625b4c5-c4cb-4d9a-a4f1-d39042c54e89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:00:22.800843 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6625b4c5-c4cb-4d9a-a4f1-d39042c54e89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:00:22.802260 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6625b4c5-c4cb-4d9a-a4f1-d39042c54e89 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:00:22.812712 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1233/2480] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:00:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:00:32.133481 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=16 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:00:32.808045 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=28 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:00:54.556592 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7faa9-a7a4-44b9-835e-6fd1b0ffa525 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:00:54.569137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7faa9-a7a4-44b9-835e-6fd1b0ffa525 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:00:54.574987 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe421e-c418-496a-b028-698ee3599db0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:00:54.575595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe421e-c418-496a-b028-698ee3599db0 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:00:54.575874 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe421e-c418-496a-b028-698ee3599db0 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:00:54.576146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe421e-c418-496a-b028-698ee3599db0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:00:54.584742 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe421e-c418-496a-b028-698ee3599db0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:00:54.586376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe421e-c418-496a-b028-698ee3599db0 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:00:54.595347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1248/2481] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:00:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:01:04.591030 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=6 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:01:54.124114 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357be916-f409-447c-83ce-cce4639af000 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:01:54.124585 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357be916-f409-447c-83ce-cce4639af000 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:01:54.124691 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357be916-f409-447c-83ce-cce4639af000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:01:54.183660 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-357be916-f409-447c-83ce-cce4639af000 None None] MFA Rules not processed for user `3a721708c7e443e799349add27de8f84`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:01:54.184854 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-357be916-f409-447c-83ce-cce4639af000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:01:54.247668 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1234/2482] 10.222.0.47 () {58 vars in 967 bytes} [Thu Aug 4 05:01:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:01:54.293977 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe421e-c418-496a-b028-698ee3599db0 service nova] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:01:54.295365 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-53fe421e-c418-496a-b028-698ee3599db0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:01:54.382986 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe421e-c418-496a-b028-698ee3599db0 service nova] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:01:54.387932 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76883e0-1da1-4156-be25-158c4c5ded71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1e539006166434790823b839bd745d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:01:54.388534 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76883e0-1da1-4156-be25-158c4c5ded71 service swift] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:01:54.388812 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76883e0-1da1-4156-be25-158c4c5ded71 service swift] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:01:54.389084 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76883e0-1da1-4156-be25-158c4c5ded71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:01:54.396939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76883e0-1da1-4156-be25-158c4c5ded71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:01:54.398527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76883e0-1da1-4156-be25-158c4c5ded71 service swift] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:01:54.406250 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1249/2483] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Aug 4 05:01:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 05:01:54.965706 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357be916-f409-447c-83ce-cce4639af000 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:01:54.976214 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357be916-f409-447c-83ce-cce4639af000 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:01:54.979968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7faa88-a710-417f-9897-be6387d34cbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:01:54.980655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7faa88-a710-417f-9897-be6387d34cbc service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:01:54.981007 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7faa88-a710-417f-9897-be6387d34cbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:01:54.981289 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7faa88-a710-417f-9897-be6387d34cbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:01:54.983083 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-8e7faa88-a710-417f-9897-be6387d34cbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:01:55.068190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7faa88-a710-417f-9897-be6387d34cbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:01:55.069537 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7faa88-a710-417f-9897-be6387d34cbc service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:01:55.089068 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1235/2484] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:01:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:01:56.320750 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76883e0-1da1-4156-be25-158c4c5ded71 service swift] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:01:56.330327 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76883e0-1da1-4156-be25-158c4c5ded71 service swift] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:01:56.334733 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ab6b8-245b-43ff-82b5-4492856e1a7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:01:56.335343 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ab6b8-245b-43ff-82b5-4492856e1a7d service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:01:56.335620 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ab6b8-245b-43ff-82b5-4492856e1a7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:01:56.335890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ab6b8-245b-43ff-82b5-4492856e1a7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:01:56.344531 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ab6b8-245b-43ff-82b5-4492856e1a7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:01:56.346146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ab6b8-245b-43ff-82b5-4492856e1a7d service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:01:56.355380 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1250/2485] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:01:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:01:58.305196 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7faa88-a710-417f-9897-be6387d34cbc service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:01:58.322164 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7faa88-a710-417f-9897-be6387d34cbc service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:01:58.336815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6d1ff-509a-4d29-923d-17d2aba64a35 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350f72f740f749f68175c1c4e8ae9d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:01:58.338553 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6d1ff-509a-4d29-923d-17d2aba64a35 service placement] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:01:58.339022 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6d1ff-509a-4d29-923d-17d2aba64a35 service placement] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:01:58.339452 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6d1ff-509a-4d29-923d-17d2aba64a35 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:01:58.343817 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-85c6d1ff-509a-4d29-923d-17d2aba64a35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:01:58.372667 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c6d1ff-509a-4d29-923d-17d2aba64a35 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:01:58.376312 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c6d1ff-509a-4d29-923d-17d2aba64a35 service placement] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:01:58.404254 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1236/2486] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:01:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:02:01.733383 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ab6b8-245b-43ff-82b5-4492856e1a7d service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:02:01.747447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ab6b8-245b-43ff-82b5-4492856e1a7d service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:02:01.754377 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c3d8e-e1ec-4dc8-a2f0-67e337c32e2b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350f72f740f749f68175c1c4e8ae9d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:02:01.755102 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c3d8e-e1ec-4dc8-a2f0-67e337c32e2b service placement] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:02:01.755658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c3d8e-e1ec-4dc8-a2f0-67e337c32e2b service placement] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:02:01.756008 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c3d8e-e1ec-4dc8-a2f0-67e337c32e2b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:02:01.758153 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-634c3d8e-e1ec-4dc8-a2f0-67e337c32e2b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:02:01.771678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634c3d8e-e1ec-4dc8-a2f0-67e337c32e2b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:02:01.774990 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634c3d8e-e1ec-4dc8-a2f0-67e337c32e2b service placement] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:02:01.786736 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6d1ff-509a-4d29-923d-17d2aba64a35 service placement] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:02:01.788165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1251/2487] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:02:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:02:01.789683 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-85c6d1ff-509a-4d29-923d-17d2aba64a35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:02:01.808079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6d1ff-509a-4d29-923d-17d2aba64a35 service placement] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:02:01.814177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbc759-acac-4021-a44b-b6b3bff31a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:02:01.814834 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbc759-acac-4021-a44b-b6b3bff31a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:02:01.815116 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbc759-acac-4021-a44b-b6b3bff31a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:02:01.815390 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbc759-acac-4021-a44b-b6b3bff31a61 admin admin] PATH_INFO: `/v3/users/2949f24248034487ada1e11bc9a9e4d7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:02:01.816599 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cbc759-acac-4021-a44b-b6b3bff31a61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2949f24248034487ada1e11bc9a9e4d7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:02:01.818717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cbc759-acac-4021-a44b-b6b3bff31a61 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:02:01.831140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:02:01.875943 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-17cbc759-acac-4021-a44b-b6b3bff31a61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2949f24248034487ada1e11bc9a9e4d7'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:02:01.876604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-17cbc759-acac-4021-a44b-b6b3bff31a61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2949f24248034487ada1e11bc9a9e4d7'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:02:01.889320 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-17cbc759-acac-4021-a44b-b6b3bff31a61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2949f24248034487ada1e11bc9a9e4d7'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:02:01.896769 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-17cbc759-acac-4021-a44b-b6b3bff31a61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2949f24248034487ada1e11bc9a9e4d7'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:02:01.897365 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-17cbc759-acac-4021-a44b-b6b3bff31a61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2949f24248034487ada1e11bc9a9e4d7'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:02:01.910037 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1237/2488] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:02:01 2022] DELETE /identity/v3/users/2949f24248034487ada1e11bc9a9e4d7 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:02:02.099874 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c3d8e-e1ec-4dc8-a2f0-67e337c32e2b service placement] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:02:02.113794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c3d8e-e1ec-4dc8-a2f0-67e337c32e2b service placement] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:02:02.173746 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b0e4b-c812-49dd-9f6b-7c7fcae82a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:02:02.174308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b0e4b-c812-49dd-9f6b-7c7fcae82a67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:02:02.174588 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b0e4b-c812-49dd-9f6b-7c7fcae82a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:02:02.174952 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b0e4b-c812-49dd-9f6b-7c7fcae82a67 admin admin] PATH_INFO: `/v3/projects/bd7fc43776684ea99eb660966ed97695` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:02:02.185360 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b0e4b-c812-49dd-9f6b-7c7fcae82a67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd7fc43776684ea99eb660966ed97695)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:02:02.187340 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b0e4b-c812-49dd-9f6b-7c7fcae82a67 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:02:02.225522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:02:02.235947 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-892b0e4b-c812-49dd-9f6b-7c7fcae82a67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd7fc43776684ea99eb660966ed97695', 'request_id': 'req-892b0e4b-c812-49dd-9f6b-7c7fcae82a67'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:02:02.245093 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-892b0e4b-c812-49dd-9f6b-7c7fcae82a67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd7fc43776684ea99eb660966ed97695', 'request_id': 'req-892b0e4b-c812-49dd-9f6b-7c7fcae82a67'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:02:02.252928 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-892b0e4b-c812-49dd-9f6b-7c7fcae82a67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd7fc43776684ea99eb660966ed97695', 'request_id': 'req-892b0e4b-c812-49dd-9f6b-7c7fcae82a67'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:02:02.264940 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-892b0e4b-c812-49dd-9f6b-7c7fcae82a67 admin admin] The token cache is being invalidate because project bd7fc43776684ea99eb660966ed97695 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:02:02.265317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-892b0e4b-c812-49dd-9f6b-7c7fcae82a67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:02:02.266646 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-892b0e4b-c812-49dd-9f6b-7c7fcae82a67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:02:02.268339 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1252/2489] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:02:02 2022] DELETE /identity/v3/projects/bd7fc43776684ea99eb660966ed97695 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:02:02.272222 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbc759-acac-4021-a44b-b6b3bff31a61 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:02:02.273645 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-17cbc759-acac-4021-a44b-b6b3bff31a61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:02:02.287160 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbc759-acac-4021-a44b-b6b3bff31a61 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:02:02.345942 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79363195-aa14-4f6c-a1dd-091184252cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:02:02.346581 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79363195-aa14-4f6c-a1dd-091184252cfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:02:02.346858 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79363195-aa14-4f6c-a1dd-091184252cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:02:02.347129 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79363195-aa14-4f6c-a1dd-091184252cfb admin admin] PATH_INFO: `/v3/users/e6f24df0f2b24487970afa57fc91363c` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:02:02.360965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79363195-aa14-4f6c-a1dd-091184252cfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6f24df0f2b24487970afa57fc91363c)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:02:02.362927 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79363195-aa14-4f6c-a1dd-091184252cfb admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:02:02.411478 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-79363195-aa14-4f6c-a1dd-091184252cfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6f24df0f2b24487970afa57fc91363c'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:02:02.412132 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-79363195-aa14-4f6c-a1dd-091184252cfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6f24df0f2b24487970afa57fc91363c'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:02:02.425082 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-79363195-aa14-4f6c-a1dd-091184252cfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6f24df0f2b24487970afa57fc91363c'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:02:02.432286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-79363195-aa14-4f6c-a1dd-091184252cfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6f24df0f2b24487970afa57fc91363c'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:02:02.432943 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-79363195-aa14-4f6c-a1dd-091184252cfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6f24df0f2b24487970afa57fc91363c'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:02:02.438765 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:02:02.448743 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1238/2490] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:02:02 2022] DELETE /identity/v3/users/e6f24df0f2b24487970afa57fc91363c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:02:02.627801 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b0e4b-c812-49dd-9f6b-7c7fcae82a67 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:02:02.629368 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-892b0e4b-c812-49dd-9f6b-7c7fcae82a67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:02:02.644643 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b0e4b-c812-49dd-9f6b-7c7fcae82a67 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:02:02.701769 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37672ef-0687-496e-a502-0a0734beb50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:02:02.702330 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37672ef-0687-496e-a502-0a0734beb50c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:02:02.702606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37672ef-0687-496e-a502-0a0734beb50c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:02:02.702875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37672ef-0687-496e-a502-0a0734beb50c admin admin] PATH_INFO: `/v3/projects/c241af3afd124c6d87f0979bc3746e59` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:02:02.713989 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37672ef-0687-496e-a502-0a0734beb50c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c241af3afd124c6d87f0979bc3746e59)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:02:02.715912 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37672ef-0687-496e-a502-0a0734beb50c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:02:02.760314 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a37672ef-0687-496e-a502-0a0734beb50c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c241af3afd124c6d87f0979bc3746e59', 'request_id': 'req-a37672ef-0687-496e-a502-0a0734beb50c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:02:02.768195 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a37672ef-0687-496e-a502-0a0734beb50c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c241af3afd124c6d87f0979bc3746e59', 'request_id': 'req-a37672ef-0687-496e-a502-0a0734beb50c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:02:02.773785 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a37672ef-0687-496e-a502-0a0734beb50c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c241af3afd124c6d87f0979bc3746e59', 'request_id': 'req-a37672ef-0687-496e-a502-0a0734beb50c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:02:02.785106 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a37672ef-0687-496e-a502-0a0734beb50c admin admin] The token cache is being invalidate because project c241af3afd124c6d87f0979bc3746e59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:02:02.785396 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a37672ef-0687-496e-a502-0a0734beb50c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:02:02.786059 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a37672ef-0687-496e-a502-0a0734beb50c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:02:02.787556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1253/2491] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:02:02 2022] DELETE /identity/v3/projects/c241af3afd124c6d87f0979bc3746e59 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:02:12.448031 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:02:12.782873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=66,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:02:20.614767 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79363195-aa14-4f6c-a1dd-091184252cfb admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:02:20.616198 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-79363195-aa14-4f6c-a1dd-091184252cfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:02:20.685634 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79363195-aa14-4f6c-a1dd-091184252cfb admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:02:20.743242 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c0dd3-0022-4972-9758-0e5aedddbcc4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350f72f740f749f68175c1c4e8ae9d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:02:20.743856 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c0dd3-0022-4972-9758-0e5aedddbcc4 service placement] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:02:20.744128 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c0dd3-0022-4972-9758-0e5aedddbcc4 service placement] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:02:20.744396 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c0dd3-0022-4972-9758-0e5aedddbcc4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:02:20.746165 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d26c0dd3-0022-4972-9758-0e5aedddbcc4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:02:20.757516 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c0dd3-0022-4972-9758-0e5aedddbcc4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:02:20.759024 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c0dd3-0022-4972-9758-0e5aedddbcc4 service placement] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:02:20.769000 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1239/2492] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:02:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:02:30.764500 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=58 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:03:58.346223 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37672ef-0687-496e-a502-0a0734beb50c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:03:58.348704 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a37672ef-0687-496e-a502-0a0734beb50c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:03:58.437047 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37672ef-0687-496e-a502-0a0734beb50c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:03:58.442734 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897e571-5aee-4c1f-977f-88af6ab34cf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:03:58.443415 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897e571-5aee-4c1f-977f-88af6ab34cf0 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:03:58.443634 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897e571-5aee-4c1f-977f-88af6ab34cf0 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:03:58.443905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897e571-5aee-4c1f-977f-88af6ab34cf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:03:58.445669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e897e571-5aee-4c1f-977f-88af6ab34cf0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:03:58.458575 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897e571-5aee-4c1f-977f-88af6ab34cf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:03:58.460150 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897e571-5aee-4c1f-977f-88af6ab34cf0 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:03:58.470515 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1254/2493] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:03:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:04:08.466274 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=39 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:04:13.928250 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c0dd3-0022-4972-9758-0e5aedddbcc4 service placement] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:04:13.929950 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d26c0dd3-0022-4972-9758-0e5aedddbcc4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:04:13.999485 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c0dd3-0022-4972-9758-0e5aedddbcc4 service placement] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:04:14.003130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e92722-a7e8-4b17-88b1-d5af2617b2b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:04:14.003911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e92722-a7e8-4b17-88b1-d5af2617b2b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:04:14.004727 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e92722-a7e8-4b17-88b1-d5af2617b2b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:04:14.005297 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e92722-a7e8-4b17-88b1-d5af2617b2b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:04:14.007194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-16e92722-a7e8-4b17-88b1-d5af2617b2b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:04:14.072845 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e92722-a7e8-4b17-88b1-d5af2617b2b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:04:14.074408 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e92722-a7e8-4b17-88b1-d5af2617b2b2 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:04:14.084078 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1240/2494] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:04:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 05:04:17.991639 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897e571-5aee-4c1f-977f-88af6ab34cf0 service nova] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:04:17.993301 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e897e571-5aee-4c1f-977f-88af6ab34cf0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:04:18.011533 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897e571-5aee-4c1f-977f-88af6ab34cf0 service nova] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:04:18.016274 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a923a6-9d05-4155-ad25-72d341282081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:04:18.016955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a923a6-9d05-4155-ad25-72d341282081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:04:18.017232 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a923a6-9d05-4155-ad25-72d341282081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:04:18.017512 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a923a6-9d05-4155-ad25-72d341282081 admin admin] PATH_INFO: `/v3/users/12f7b1c19d824ac7b8207bf001b79057` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:04:18.033373 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a923a6-9d05-4155-ad25-72d341282081 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12f7b1c19d824ac7b8207bf001b79057)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:04:18.035356 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a923a6-9d05-4155-ad25-72d341282081 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:04:18.089889 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-10a923a6-9d05-4155-ad25-72d341282081 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12f7b1c19d824ac7b8207bf001b79057'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:04:18.097903 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-10a923a6-9d05-4155-ad25-72d341282081 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12f7b1c19d824ac7b8207bf001b79057'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:04:18.099033 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-10a923a6-9d05-4155-ad25-72d341282081 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12f7b1c19d824ac7b8207bf001b79057'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:04:18.112867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-10a923a6-9d05-4155-ad25-72d341282081 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12f7b1c19d824ac7b8207bf001b79057'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:04:18.113846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-10a923a6-9d05-4155-ad25-72d341282081 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12f7b1c19d824ac7b8207bf001b79057'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:04:18.127338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1255/2495] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:04:17 2022] DELETE /identity/v3/users/12f7b1c19d824ac7b8207bf001b79057 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:04:18.306541 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e92722-a7e8-4b17-88b1-d5af2617b2b2 service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:04:18.317155 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e92722-a7e8-4b17-88b1-d5af2617b2b2 service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:04:18.375308 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e867277-9d10-477e-9bd9-907fbbe3b856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:04:18.375948 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e867277-9d10-477e-9bd9-907fbbe3b856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:04:18.376230 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e867277-9d10-477e-9bd9-907fbbe3b856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:04:18.376502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e867277-9d10-477e-9bd9-907fbbe3b856 admin admin] PATH_INFO: `/v3/projects/157a41f0f76c41cb8dfff71a14138534` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:04:18.377731 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e867277-9d10-477e-9bd9-907fbbe3b856 admin admin] RBAC: Authorizing `identity:delete_project(project_id=157a41f0f76c41cb8dfff71a14138534)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:04:18.380483 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e867277-9d10-477e-9bd9-907fbbe3b856 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:04:18.426584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5e867277-9d10-477e-9bd9-907fbbe3b856 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '157a41f0f76c41cb8dfff71a14138534', 'request_id': 'req-5e867277-9d10-477e-9bd9-907fbbe3b856'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:04:18.428840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:04:18.433731 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5e867277-9d10-477e-9bd9-907fbbe3b856 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '157a41f0f76c41cb8dfff71a14138534', 'request_id': 'req-5e867277-9d10-477e-9bd9-907fbbe3b856'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:04:18.440101 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5e867277-9d10-477e-9bd9-907fbbe3b856 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '157a41f0f76c41cb8dfff71a14138534', 'request_id': 'req-5e867277-9d10-477e-9bd9-907fbbe3b856'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:04:18.451660 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5e867277-9d10-477e-9bd9-907fbbe3b856 admin admin] The token cache is being invalidate because project 157a41f0f76c41cb8dfff71a14138534 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:04:18.451948 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5e867277-9d10-477e-9bd9-907fbbe3b856 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:04:18.452587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5e867277-9d10-477e-9bd9-907fbbe3b856 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:04:18.454128 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1241/2496] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:04:18 2022] DELETE /identity/v3/projects/157a41f0f76c41cb8dfff71a14138534 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:04:18.460081 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71cfa7-0021-45bd-a25e-8506ff30d2f8 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:04:18.460375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71cfa7-0021-45bd-a25e-8506ff30d2f8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:04:18.460651 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71cfa7-0021-45bd-a25e-8506ff30d2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:04:18.518101 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-1f71cfa7-0021-45bd-a25e-8506ff30d2f8 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:04:18.519905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1f71cfa7-0021-45bd-a25e-8506ff30d2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:04:18.577236 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1256/2497] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:04:18.585531 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672a1b3-ecc6-46e5-a913-d4cf8f7cc0c7 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:04:18.585887 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672a1b3-ecc6-46e5-a913-d4cf8f7cc0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:04:18.586134 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672a1b3-ecc6-46e5-a913-d4cf8f7cc0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:04:18.606288 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-f672a1b3-ecc6-46e5-a913-d4cf8f7cc0c7 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:04:18.607489 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-f672a1b3-ecc6-46e5-a913-d4cf8f7cc0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:04:18.616029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1242/2498] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:04:18.620228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71cfa7-0021-45bd-a25e-8506ff30d2f8 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:04:18.630791 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71cfa7-0021-45bd-a25e-8506ff30d2f8 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:04:18.635436 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853d683-530e-45b2-8ff6-0a745860c24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:04:18.636044 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853d683-530e-45b2-8ff6-0a745860c24a admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:04:18.636329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853d683-530e-45b2-8ff6-0a745860c24a admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:04:18.636606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853d683-530e-45b2-8ff6-0a745860c24a admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:04:18.637303 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0853d683-530e-45b2-8ff6-0a745860c24a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 05:04:18.637639 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0853d683-530e-45b2-8ff6-0a745860c24a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:04:18.639232 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0853d683-530e-45b2-8ff6-0a745860c24a admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:04:18.650664 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1257/2499] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 05:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 05:04:18.654805 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672a1b3-ecc6-46e5-a913-d4cf8f7cc0c7 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:04:18.663839 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672a1b3-ecc6-46e5-a913-d4cf8f7cc0c7 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:04:18.668668 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e2e1a3-4ae5-437a-b987-29edaf2bcfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:04:18.669271 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2e1a3-4ae5-437a-b987-29edaf2bcfd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:04:18.669599 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2e1a3-4ae5-437a-b987-29edaf2bcfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:04:18.669873 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2e1a3-4ae5-437a-b987-29edaf2bcfd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:04:18.670809 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e2e1a3-4ae5-437a-b987-29edaf2bcfd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:04:18.672736 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e2e1a3-4ae5-437a-b987-29edaf2bcfd0 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:04:18.713587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1243/2500] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:04:18 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 05:04:18.714268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853d683-530e-45b2-8ff6-0a745860c24a admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:04:18.726166 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853d683-530e-45b2-8ff6-0a745860c24a admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:04:18.737945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:04:18.784303 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf13dc2-f415-4efe-890b-d6978e20bd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:04:18.785405 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf13dc2-f415-4efe-890b-d6978e20bd68 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:04:18.785701 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf13dc2-f415-4efe-890b-d6978e20bd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:04:18.785973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf13dc2-f415-4efe-890b-d6978e20bd68 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:04:18.786638 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf13dc2-f415-4efe-890b-d6978e20bd68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:04:18.788530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf13dc2-f415-4efe-890b-d6978e20bd68 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:04:18.809491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1258/2501] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:04:18 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 05:04:18.813677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e2e1a3-4ae5-437a-b987-29edaf2bcfd0 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:04:18.824016 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e2e1a3-4ae5-437a-b987-29edaf2bcfd0 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:04:18.828687 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e4231-79b2-413a-a879-2d17c2bad092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:04:18.829281 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e4231-79b2-413a-a879-2d17c2bad092 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:04:18.829579 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e4231-79b2-413a-a879-2d17c2bad092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:04:18.829858 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e4231-79b2-413a-a879-2d17c2bad092 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:04:18.830546 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e4231-79b2-413a-a879-2d17c2bad092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:04:18.832454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e4231-79b2-413a-a879-2d17c2bad092 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:04:18.842339 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1244/2502] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 05:04:18.846068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf13dc2-f415-4efe-890b-d6978e20bd68 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:04:18.856445 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf13dc2-f415-4efe-890b-d6978e20bd68 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:04:18.860982 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1fe77-b45a-4118-a3f3-ac4a7e5e03d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:04:18.861607 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1fe77-b45a-4118-a3f3-ac4a7e5e03d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:04:18.861890 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1fe77-b45a-4118-a3f3-ac4a7e5e03d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:04:18.862159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1fe77-b45a-4118-a3f3-ac4a7e5e03d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:04:18.862847 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f1fe77-b45a-4118-a3f3-ac4a7e5e03d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:04:18.865507 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f1fe77-b45a-4118-a3f3-ac4a7e5e03d9 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:04:18.875914 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1259/2503] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:04:18.880298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e4231-79b2-413a-a879-2d17c2bad092 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:04:18.890051 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e4231-79b2-413a-a879-2d17c2bad092 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:04:18.894579 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b576d-08ce-4e41-93a6-63236de89797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:04:18.895217 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b576d-08ce-4e41-93a6-63236de89797 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:04:18.895496 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b576d-08ce-4e41-93a6-63236de89797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:04:18.895769 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b576d-08ce-4e41-93a6-63236de89797 admin admin] PATH_INFO: `/v3/projects/1f07f2e1a0d14749be23ba5fadb2e2de/users/0c4d68028f274f1aa98f43d259df16c4/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:04:18.911163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b576d-08ce-4e41-93a6-63236de89797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f07f2e1a0d14749be23ba5fadb2e2de, user_id=0c4d68028f274f1aa98f43d259df16c4, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:04:18.913864 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b576d-08ce-4e41-93a6-63236de89797 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:04:18.926898 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1245/2504] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:04:18 2022] PUT /identity/v3/projects/1f07f2e1a0d14749be23ba5fadb2e2de/users/0c4d68028f274f1aa98f43d259df16c4/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 05:04:18.938405 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f13a9d-03d5-405d-92d2-0babd1e7cdce None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:04:18.938691 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f13a9d-03d5-405d-92d2-0babd1e7cdce None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:04:18.938965 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f13a9d-03d5-405d-92d2-0babd1e7cdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:04:18.966366 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-83f13a9d-03d5-405d-92d2-0babd1e7cdce None None] MFA Rules not processed for user `0c4d68028f274f1aa98f43d259df16c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:04:18.967500 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-83f13a9d-03d5-405d-92d2-0babd1e7cdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:04:19.029315 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1260/2505] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:04:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:04:19.041143 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b576d-08ce-4e41-93a6-63236de89797 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:04:19.042871 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-651b576d-08ce-4e41-93a6-63236de89797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:04:19.115508 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b576d-08ce-4e41-93a6-63236de89797 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:04:19.120964 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47004dd3-2159-47aa-8caa-02040f622399 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:04:19.121591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47004dd3-2159-47aa-8caa-02040f622399 service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:04:19.121894 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47004dd3-2159-47aa-8caa-02040f622399 service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:04:19.122164 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47004dd3-2159-47aa-8caa-02040f622399 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:04:19.129893 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47004dd3-2159-47aa-8caa-02040f622399 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:04:19.131480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47004dd3-2159-47aa-8caa-02040f622399 service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:04:19.140560 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1246/2506] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:04:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:04:29.026517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=58,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:04:29.136419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:DELETE=3,keystone:SELECT=71,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:05:18.368869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f13a9d-03d5-405d-92d2-0babd1e7cdce None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:05:18.370453 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-83f13a9d-03d5-405d-92d2-0babd1e7cdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:05:18.438830 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f13a9d-03d5-405d-92d2-0babd1e7cdce None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:05:18.443320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa9353-ce64-4280-b2e2-a0ce2f905e92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:05:18.443939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa9353-ce64-4280-b2e2-a0ce2f905e92 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:05:18.444217 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa9353-ce64-4280-b2e2-a0ce2f905e92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:05:18.444487 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa9353-ce64-4280-b2e2-a0ce2f905e92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:05:18.446237 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-09aa9353-ce64-4280-b2e2-a0ce2f905e92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:05:18.455519 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aa9353-ce64-4280-b2e2-a0ce2f905e92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:05:18.456930 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aa9353-ce64-4280-b2e2-a0ce2f905e92 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:05:18.464712 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1261/2507] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:05:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 05:05:28.461373 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=31 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:05:56.900244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47004dd3-2159-47aa-8caa-02040f622399 service nova] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:05:56.901848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-47004dd3-2159-47aa-8caa-02040f622399 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:05:56.976842 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47004dd3-2159-47aa-8caa-02040f622399 service nova] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:05:56.980316 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d867c1c-c0f7-4b9d-be05-b3acc9aad0eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:05:56.980925 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d867c1c-c0f7-4b9d-be05-b3acc9aad0eb service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:05:56.981200 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d867c1c-c0f7-4b9d-be05-b3acc9aad0eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:05:56.981468 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d867c1c-c0f7-4b9d-be05-b3acc9aad0eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:05:56.983201 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5d867c1c-c0f7-4b9d-be05-b3acc9aad0eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:05:57.049371 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d867c1c-c0f7-4b9d-be05-b3acc9aad0eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:05:57.050914 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d867c1c-c0f7-4b9d-be05-b3acc9aad0eb service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:05:57.069858 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1247/2508] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:05:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:05:57.118649 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa9353-ce64-4280-b2e2-a0ce2f905e92 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:05:57.127910 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa9353-ce64-4280-b2e2-a0ce2f905e92 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:05:57.132333 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bac1c9-5087-4c2f-9edb-0cbecb4ddde5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:05:57.132928 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bac1c9-5087-4c2f-9edb-0cbecb4ddde5 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:05:57.133203 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bac1c9-5087-4c2f-9edb-0cbecb4ddde5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:05:57.133471 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bac1c9-5087-4c2f-9edb-0cbecb4ddde5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:05:57.135212 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-10bac1c9-5087-4c2f-9edb-0cbecb4ddde5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:05:57.214914 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bac1c9-5087-4c2f-9edb-0cbecb4ddde5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:05:57.216352 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bac1c9-5087-4c2f-9edb-0cbecb4ddde5 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:05:57.226773 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1262/2509] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:05:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 05:06:07.066365 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=61 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:06:07.221450 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=33 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:06:14.432961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d867c1c-c0f7-4b9d-be05-b3acc9aad0eb service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:06:14.434366 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5d867c1c-c0f7-4b9d-be05-b3acc9aad0eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:06:14.445619 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d867c1c-c0f7-4b9d-be05-b3acc9aad0eb service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:06:14.448785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ee859-17a8-47f8-b5df-a338ecc9ca73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:06:14.449393 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ee859-17a8-47f8-b5df-a338ecc9ca73 service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:06:14.449698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ee859-17a8-47f8-b5df-a338ecc9ca73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:06:14.449970 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ee859-17a8-47f8-b5df-a338ecc9ca73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:06:14.452283 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d58ee859-17a8-47f8-b5df-a338ecc9ca73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:06:14.467732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ee859-17a8-47f8-b5df-a338ecc9ca73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:06:14.469145 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ee859-17a8-47f8-b5df-a338ecc9ca73 service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:06:14.480596 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1248/2510] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:06:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:06:14.618129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bac1c9-5087-4c2f-9edb-0cbecb4ddde5 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:06:14.626591 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bac1c9-5087-4c2f-9edb-0cbecb4ddde5 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:06:14.630334 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0c9cc-154b-4fd5-abe9-184be5784d4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:06:14.630944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0c9cc-154b-4fd5-abe9-184be5784d4b service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:06:14.631224 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0c9cc-154b-4fd5-abe9-184be5784d4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:06:14.631495 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0c9cc-154b-4fd5-abe9-184be5784d4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:06:14.633229 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a4e0c9cc-154b-4fd5-abe9-184be5784d4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:06:14.647227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0c9cc-154b-4fd5-abe9-184be5784d4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:06:14.648650 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0c9cc-154b-4fd5-abe9-184be5784d4b service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:06:14.659440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1263/2511] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:06:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:06:24.476464 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=6 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:06:24.654683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=6 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:07:23.605319 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ee859-17a8-47f8-b5df-a338ecc9ca73 service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:07:23.606753 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d58ee859-17a8-47f8-b5df-a338ecc9ca73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:07:23.722310 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ee859-17a8-47f8-b5df-a338ecc9ca73 service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:07:23.727830 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9bdcb0-402f-4739-af35-ba452d2c73f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350f72f740f749f68175c1c4e8ae9d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:07:23.728438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bdcb0-402f-4739-af35-ba452d2c73f8 service placement] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:07:23.728714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bdcb0-402f-4739-af35-ba452d2c73f8 service placement] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:07:23.728983 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bdcb0-402f-4739-af35-ba452d2c73f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:07:23.730745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-1c9bdcb0-402f-4739-af35-ba452d2c73f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:07:23.742435 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9bdcb0-402f-4739-af35-ba452d2c73f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:07:23.743984 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9bdcb0-402f-4739-af35-ba452d2c73f8 service placement] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:07:23.753562 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1249/2512] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:07:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 05:07:33.749501 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=54 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:07:57.326801 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0c9cc-154b-4fd5-abe9-184be5784d4b service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:07:57.332662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0c9cc-154b-4fd5-abe9-184be5784d4b service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:07:57.336805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dd1a7-5e41-43ad-b575-567d281f9dcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:07:57.337481 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dd1a7-5e41-43ad-b575-567d281f9dcc service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:07:57.337786 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dd1a7-5e41-43ad-b575-567d281f9dcc service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:07:57.338055 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dd1a7-5e41-43ad-b575-567d281f9dcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:07:57.339787 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f27dd1a7-5e41-43ad-b575-567d281f9dcc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:07:57.419644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27dd1a7-5e41-43ad-b575-567d281f9dcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:07:57.421071 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27dd1a7-5e41-43ad-b575-567d281f9dcc service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:07:57.438684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1264/2513] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:07:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:08:07.436130 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=41 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:08:14.746594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9bdcb0-402f-4739-af35-ba452d2c73f8 service placement] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:08:14.824372 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9bdcb0-402f-4739-af35-ba452d2c73f8 service placement] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:08:14.828971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3c53c-9454-410a-b190-edb41c8be063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:08:14.829655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3c53c-9454-410a-b190-edb41c8be063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:08:14.829915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3c53c-9454-410a-b190-edb41c8be063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:08:14.830189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3c53c-9454-410a-b190-edb41c8be063 admin admin] PATH_INFO: `/v3/users/0c4d68028f274f1aa98f43d259df16c4` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:08:14.831350 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3c53c-9454-410a-b190-edb41c8be063 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c4d68028f274f1aa98f43d259df16c4)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:08:14.833265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3c53c-9454-410a-b190-edb41c8be063 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:08:14.886490 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-23b3c53c-9454-410a-b190-edb41c8be063 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c4d68028f274f1aa98f43d259df16c4'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:08:14.887171 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-23b3c53c-9454-410a-b190-edb41c8be063 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c4d68028f274f1aa98f43d259df16c4'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:08:14.899368 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-23b3c53c-9454-410a-b190-edb41c8be063 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c4d68028f274f1aa98f43d259df16c4'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:08:14.906726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-23b3c53c-9454-410a-b190-edb41c8be063 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c4d68028f274f1aa98f43d259df16c4'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:08:14.907397 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-23b3c53c-9454-410a-b190-edb41c8be063 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c4d68028f274f1aa98f43d259df16c4'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:08:14.920625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1250/2514] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:08:14 2022] DELETE /identity/v3/users/0c4d68028f274f1aa98f43d259df16c4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:08:14.925288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b38273-7207-4e40-b211-d77bf191553d None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:08:14.925832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b38273-7207-4e40-b211-d77bf191553d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:08:14.925955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b38273-7207-4e40-b211-d77bf191553d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:08:14.945788 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-15b38273-7207-4e40-b211-d77bf191553d None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:08:14.947511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-15b38273-7207-4e40-b211-d77bf191553d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:08:15.005140 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1265/2515] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:08:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:08:15.015583 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3c53c-9454-410a-b190-edb41c8be063 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:08:15.017003 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-23b3c53c-9454-410a-b190-edb41c8be063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:08:15.074067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3c53c-9454-410a-b190-edb41c8be063 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:08:15.077773 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4ed00-fbee-4f36-80a0-969e5bdd4328 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:08:15.078376 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4ed00-fbee-4f36-80a0-969e5bdd4328 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:08:15.078659 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4ed00-fbee-4f36-80a0-969e5bdd4328 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:08:15.078935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4ed00-fbee-4f36-80a0-969e5bdd4328 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:08:15.088593 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e4ed00-fbee-4f36-80a0-969e5bdd4328 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:08:15.090034 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e4ed00-fbee-4f36-80a0-969e5bdd4328 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:08:15.099289 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1251/2516] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:08:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:08:15.378253 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b38273-7207-4e40-b211-d77bf191553d None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:08:15.380035 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-15b38273-7207-4e40-b211-d77bf191553d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:08:15.394501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b38273-7207-4e40-b211-d77bf191553d None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:08:15.401446 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6d0cc-d8de-423a-a646-883d4ff89e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:08:15.403121 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6d0cc-d8de-423a-a646-883d4ff89e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:08:15.403578 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6d0cc-d8de-423a-a646-883d4ff89e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:08:15.403979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6d0cc-d8de-423a-a646-883d4ff89e37 admin admin] PATH_INFO: `/v3/projects/1f07f2e1a0d14749be23ba5fadb2e2de` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:08:15.405185 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6d0cc-d8de-423a-a646-883d4ff89e37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f07f2e1a0d14749be23ba5fadb2e2de)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:08:15.407764 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6d0cc-d8de-423a-a646-883d4ff89e37 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:08:15.456246 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-98f6d0cc-d8de-423a-a646-883d4ff89e37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f07f2e1a0d14749be23ba5fadb2e2de', 'request_id': 'req-98f6d0cc-d8de-423a-a646-883d4ff89e37'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:08:15.463516 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-98f6d0cc-d8de-423a-a646-883d4ff89e37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f07f2e1a0d14749be23ba5fadb2e2de', 'request_id': 'req-98f6d0cc-d8de-423a-a646-883d4ff89e37'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:08:15.469409 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-98f6d0cc-d8de-423a-a646-883d4ff89e37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f07f2e1a0d14749be23ba5fadb2e2de', 'request_id': 'req-98f6d0cc-d8de-423a-a646-883d4ff89e37'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:08:15.480748 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-98f6d0cc-d8de-423a-a646-883d4ff89e37 admin admin] The token cache is being invalidate because project 1f07f2e1a0d14749be23ba5fadb2e2de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:08:15.481032 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-98f6d0cc-d8de-423a-a646-883d4ff89e37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:08:15.481743 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-98f6d0cc-d8de-423a-a646-883d4ff89e37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:08:15.483250 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1266/2517] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:08:15 2022] DELETE /identity/v3/projects/1f07f2e1a0d14749be23ba5fadb2e2de => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 05:08:15.488115 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76634358-9ca2-4a73-8568-46bcca132d82 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:08:15.488507 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76634358-9ca2-4a73-8568-46bcca132d82 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:08:15.488844 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76634358-9ca2-4a73-8568-46bcca132d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:08:15.509113 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-76634358-9ca2-4a73-8568-46bcca132d82 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:08:15.510400 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-76634358-9ca2-4a73-8568-46bcca132d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:08:15.527672 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:08:15.572232 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1252/2518] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:08:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 05:08:15.578371 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352eb46e-3be2-4754-9e0d-482f0388d139 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:08:15.578685 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352eb46e-3be2-4754-9e0d-482f0388d139 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:08:15.578958 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352eb46e-3be2-4754-9e0d-482f0388d139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:08:15.598586 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-352eb46e-3be2-4754-9e0d-482f0388d139 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:08:15.599767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-352eb46e-3be2-4754-9e0d-482f0388d139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:08:15.607780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1267/2519] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:08:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:08:15.611930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76634358-9ca2-4a73-8568-46bcca132d82 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:08:15.621750 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76634358-9ca2-4a73-8568-46bcca132d82 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:08:15.626263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f277b7-0483-4ae4-8c29-be2297c331ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:08:15.626863 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f277b7-0483-4ae4-8c29-be2297c331ba admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:08:15.627142 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f277b7-0483-4ae4-8c29-be2297c331ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:08:15.627416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f277b7-0483-4ae4-8c29-be2297c331ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:08:15.628102 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f277b7-0483-4ae4-8c29-be2297c331ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 05:08:15.628406 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f277b7-0483-4ae4-8c29-be2297c331ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:08:15.630038 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f277b7-0483-4ae4-8c29-be2297c331ba admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:08:15.642293 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1253/2520] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 05:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 05:08:15.646482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352eb46e-3be2-4754-9e0d-482f0388d139 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:08:15.656499 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352eb46e-3be2-4754-9e0d-482f0388d139 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:08:15.661115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f424552-c64c-4094-9e13-3d80a203654a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:08:15.661755 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f424552-c64c-4094-9e13-3d80a203654a admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:08:15.662038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f424552-c64c-4094-9e13-3d80a203654a admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:08:15.662323 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f424552-c64c-4094-9e13-3d80a203654a admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:08:15.662975 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f424552-c64c-4094-9e13-3d80a203654a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:08:15.665003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f424552-c64c-4094-9e13-3d80a203654a admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:08:15.691192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1268/2521] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:08:15 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:08:15.695375 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f277b7-0483-4ae4-8c29-be2297c331ba admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:08:15.706080 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f277b7-0483-4ae4-8c29-be2297c331ba admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:08:15.760791 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056666e-4a20-4c05-92c6-07b7d058a04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:08:15.761436 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056666e-4a20-4c05-92c6-07b7d058a04e admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:08:15.761615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056666e-4a20-4c05-92c6-07b7d058a04e admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:08:15.761890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056666e-4a20-4c05-92c6-07b7d058a04e admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:08:15.762555 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f056666e-4a20-4c05-92c6-07b7d058a04e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:08:15.764437 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f056666e-4a20-4c05-92c6-07b7d058a04e admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:08:15.785323 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1254/2522] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:08:15 2022] POST /identity/v3/users => generated 283 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:08:15.789455 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f424552-c64c-4094-9e13-3d80a203654a admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:08:15.799964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f424552-c64c-4094-9e13-3d80a203654a admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:08:15.804477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfad8ba-c637-4d1f-a8d2-a0645b0a0059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:08:15.805648 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfad8ba-c637-4d1f-a8d2-a0645b0a0059 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:08:15.805960 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfad8ba-c637-4d1f-a8d2-a0645b0a0059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:08:15.806335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfad8ba-c637-4d1f-a8d2-a0645b0a0059 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:08:15.807022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfad8ba-c637-4d1f-a8d2-a0645b0a0059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:08:15.808589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfad8ba-c637-4d1f-a8d2-a0645b0a0059 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:08:15.818167 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1269/2523] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:08:15.822665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056666e-4a20-4c05-92c6-07b7d058a04e admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:08:15.832466 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056666e-4a20-4c05-92c6-07b7d058a04e admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:08:15.837094 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d4da1-d90c-4f57-b651-dcc811460b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:08:15.837729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d4da1-d90c-4f57-b651-dcc811460b0f admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:08:15.838038 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d4da1-d90c-4f57-b651-dcc811460b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:08:15.838310 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d4da1-d90c-4f57-b651-dcc811460b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:08:15.839027 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d4da1-d90c-4f57-b651-dcc811460b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:08:15.840586 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d4da1-d90c-4f57-b651-dcc811460b0f admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:08:15.850983 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1255/2524] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 05:08:15.855146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfad8ba-c637-4d1f-a8d2-a0645b0a0059 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:08:15.865153 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfad8ba-c637-4d1f-a8d2-a0645b0a0059 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:08:15.870601 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a23a73-89e8-4baa-8be6-c6b65d77d7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:08:15.871256 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a23a73-89e8-4baa-8be6-c6b65d77d7f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:08:15.871562 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a23a73-89e8-4baa-8be6-c6b65d77d7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:08:15.871844 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a23a73-89e8-4baa-8be6-c6b65d77d7f6 admin admin] PATH_INFO: `/v3/projects/922768dc24754a8a8ea7c774fe2ccb45/users/af595b52687f4027b505adf49e1ebffc/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:08:15.886953 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a23a73-89e8-4baa-8be6-c6b65d77d7f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=922768dc24754a8a8ea7c774fe2ccb45, user_id=af595b52687f4027b505adf49e1ebffc, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:08:15.889612 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a23a73-89e8-4baa-8be6-c6b65d77d7f6 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:08:15.903173 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1270/2525] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:08:15 2022] PUT /identity/v3/projects/922768dc24754a8a8ea7c774fe2ccb45/users/af595b52687f4027b505adf49e1ebffc/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:08:15.913263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc38a5-9d43-4eb5-9171-df4968b1d8a3 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:08:15.913568 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc38a5-9d43-4eb5-9171-df4968b1d8a3 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:08:15.913864 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc38a5-9d43-4eb5-9171-df4968b1d8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:08:15.942254 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-18fc38a5-9d43-4eb5-9171-df4968b1d8a3 None None] MFA Rules not processed for user `af595b52687f4027b505adf49e1ebffc`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:08:15.943470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-18fc38a5-9d43-4eb5-9171-df4968b1d8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:08:16.006512 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1256/2526] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:08:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:08:16.011907 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a23a73-89e8-4baa-8be6-c6b65d77d7f6 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:08:16.032517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:08:16.075343 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a23a73-89e8-4baa-8be6-c6b65d77d7f6 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:08:16.080174 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923ddb2-93f3-4dac-a8f3-aadb81d0d0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:08:16.080782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923ddb2-93f3-4dac-a8f3-aadb81d0d0be admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:08:16.081087 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923ddb2-93f3-4dac-a8f3-aadb81d0d0be admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:08:16.081357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923ddb2-93f3-4dac-a8f3-aadb81d0d0be admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:08:16.082078 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9923ddb2-93f3-4dac-a8f3-aadb81d0d0be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:08:16.083953 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9923ddb2-93f3-4dac-a8f3-aadb81d0d0be admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:08:16.109320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1271/2527] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:08:16 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 05:08:16.113434 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fc38a5-9d43-4eb5-9171-df4968b1d8a3 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:08:16.124601 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fc38a5-9d43-4eb5-9171-df4968b1d8a3 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:08:16.137264 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:08:16.181896 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37811be-d8d6-47ed-bce8-89c9b81a0828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:08:16.182507 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37811be-d8d6-47ed-bce8-89c9b81a0828 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:08:16.182783 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37811be-d8d6-47ed-bce8-89c9b81a0828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:08:16.183050 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37811be-d8d6-47ed-bce8-89c9b81a0828 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:08:16.183714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37811be-d8d6-47ed-bce8-89c9b81a0828 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:08:16.185697 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37811be-d8d6-47ed-bce8-89c9b81a0828 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:08:16.205420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1257/2528] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:08:16 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:08:16.209347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923ddb2-93f3-4dac-a8f3-aadb81d0d0be admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:08:16.219411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923ddb2-93f3-4dac-a8f3-aadb81d0d0be admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:08:16.223921 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbea3a-deed-46a6-8fe6-d3888c797ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:08:16.224518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbea3a-deed-46a6-8fe6-d3888c797ffe admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:08:16.224791 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbea3a-deed-46a6-8fe6-d3888c797ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:08:16.225062 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbea3a-deed-46a6-8fe6-d3888c797ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:08:16.225782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbea3a-deed-46a6-8fe6-d3888c797ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:08:16.227992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbea3a-deed-46a6-8fe6-d3888c797ffe admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:08:16.238251 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1272/2529] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:08:16.242338 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37811be-d8d6-47ed-bce8-89c9b81a0828 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:08:16.252760 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37811be-d8d6-47ed-bce8-89c9b81a0828 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:08:16.257334 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972512e-f7f1-44a3-a4f2-4cab0191d287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:08:16.258015 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972512e-f7f1-44a3-a4f2-4cab0191d287 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:08:16.258359 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972512e-f7f1-44a3-a4f2-4cab0191d287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:08:16.258627 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972512e-f7f1-44a3-a4f2-4cab0191d287 admin admin] PATH_INFO: `/v3/projects/be441fa5c8274b4ab7baf52de8a1e644/users/fb61e0e45ea44a2582d0bdab866c9bb3/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:08:16.274810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9972512e-f7f1-44a3-a4f2-4cab0191d287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be441fa5c8274b4ab7baf52de8a1e644, user_id=fb61e0e45ea44a2582d0bdab866c9bb3, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:08:16.277499 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9972512e-f7f1-44a3-a4f2-4cab0191d287 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:08:16.292064 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1258/2530] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:08:16 2022] PUT /identity/v3/projects/be441fa5c8274b4ab7baf52de8a1e644/users/fb61e0e45ea44a2582d0bdab866c9bb3/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 05:08:16.301679 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbea3a-deed-46a6-8fe6-d3888c797ffe admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:08:16.309448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbea3a-deed-46a6-8fe6-d3888c797ffe admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:08:16.364134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9ed62-9fb0-4eee-a74f-82b01da24757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:08:16.364774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9ed62-9fb0-4eee-a74f-82b01da24757 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:08:16.365085 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9ed62-9fb0-4eee-a74f-82b01da24757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:08:16.365357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9ed62-9fb0-4eee-a74f-82b01da24757 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:08:16.366086 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9ed62-9fb0-4eee-a74f-82b01da24757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:08:16.368791 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9ed62-9fb0-4eee-a74f-82b01da24757 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:08:16.378552 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1273/2531] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:08:16.382720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972512e-f7f1-44a3-a4f2-4cab0191d287 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:08:16.393103 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972512e-f7f1-44a3-a4f2-4cab0191d287 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:08:16.397713 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96673deb-6915-4ab8-8c92-c0d3e2548c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:08:16.398387 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96673deb-6915-4ab8-8c92-c0d3e2548c2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:08:16.398691 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96673deb-6915-4ab8-8c92-c0d3e2548c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:08:16.398975 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96673deb-6915-4ab8-8c92-c0d3e2548c2b admin admin] PATH_INFO: `/v3/projects/be441fa5c8274b4ab7baf52de8a1e644/users/fb61e0e45ea44a2582d0bdab866c9bb3/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:08:16.400946 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96673deb-6915-4ab8-8c92-c0d3e2548c2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=be441fa5c8274b4ab7baf52de8a1e644, user_id=fb61e0e45ea44a2582d0bdab866c9bb3, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:08:16.403547 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96673deb-6915-4ab8-8c92-c0d3e2548c2b admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:08:16.413684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1259/2532] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:08:16 2022] PUT /identity/v3/projects/be441fa5c8274b4ab7baf52de8a1e644/users/fb61e0e45ea44a2582d0bdab866c9bb3/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:08:16.424568 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8725f9b-e7da-4c0d-910b-2c9b5bf767a8 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:08:16.424856 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8725f9b-e7da-4c0d-910b-2c9b5bf767a8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:08:16.425129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8725f9b-e7da-4c0d-910b-2c9b5bf767a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:08:16.452874 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-f8725f9b-e7da-4c0d-910b-2c9b5bf767a8 None None] MFA Rules not processed for user `fb61e0e45ea44a2582d0bdab866c9bb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:08:16.454104 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f8725f9b-e7da-4c0d-910b-2c9b5bf767a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:08:16.522779 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1274/2533] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:08:16 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:08:26.409855 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=36,keystone:INSERT=5 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:08:26.519837 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=92,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:09:57.992621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96673deb-6915-4ab8-8c92-c0d3e2548c2b admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:09:57.994474 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-96673deb-6915-4ab8-8c92-c0d3e2548c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:09:58.071471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96673deb-6915-4ab8-8c92-c0d3e2548c2b admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:09:58.075096 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244be65-c74b-496a-944c-c29b306fca8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:09:58.075694 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244be65-c74b-496a-944c-c29b306fca8b service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:09:58.075969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244be65-c74b-496a-944c-c29b306fca8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:09:58.076236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244be65-c74b-496a-944c-c29b306fca8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:09:58.086187 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244be65-c74b-496a-944c-c29b306fca8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:09:58.087582 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244be65-c74b-496a-944c-c29b306fca8b service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:09:58.096689 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1260/2534] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:09:57 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:10:08.092629 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=35 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:10:24.342619 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8725f9b-e7da-4c0d-910b-2c9b5bf767a8 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:10:24.344123 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f8725f9b-e7da-4c0d-910b-2c9b5bf767a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:10:24.427735 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8725f9b-e7da-4c0d-910b-2c9b5bf767a8 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:10:24.431517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab2ea7-0c44-4f65-b756-54feb73046b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:10:24.432125 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab2ea7-0c44-4f65-b756-54feb73046b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:10:24.432405 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab2ea7-0c44-4f65-b756-54feb73046b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:10:24.432678 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab2ea7-0c44-4f65-b756-54feb73046b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:10:24.434444 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b0ab2ea7-0c44-4f65-b756-54feb73046b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:10:24.444011 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab2ea7-0c44-4f65-b756-54feb73046b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:10:24.445433 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab2ea7-0c44-4f65-b756-54feb73046b9 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:10:24.453645 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1275/2535] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:10:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:10:34.449856 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=36 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:11:58.392916 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244be65-c74b-496a-944c-c29b306fca8b service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:11:58.402173 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244be65-c74b-496a-944c-c29b306fca8b service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:11:58.405747 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70aa478-3f3e-4376-8e1c-2749718c5253 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:11:58.406337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70aa478-3f3e-4376-8e1c-2749718c5253 service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:11:58.406613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70aa478-3f3e-4376-8e1c-2749718c5253 service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:11:58.406901 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70aa478-3f3e-4376-8e1c-2749718c5253 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:11:58.408611 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-b70aa478-3f3e-4376-8e1c-2749718c5253 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:11:58.478432 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70aa478-3f3e-4376-8e1c-2749718c5253 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:11:58.479852 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70aa478-3f3e-4376-8e1c-2749718c5253 service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:11:58.489474 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1261/2536] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:11:58 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:11:58.547328 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab2ea7-0c44-4f65-b756-54feb73046b9 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:11:58.557004 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab2ea7-0c44-4f65-b756-54feb73046b9 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:11:58.561055 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542b270-a175-4b43-9295-8096d4eb4092 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:11:58.561705 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542b270-a175-4b43-9295-8096d4eb4092 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:11:58.562148 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542b270-a175-4b43-9295-8096d4eb4092 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:11:58.562513 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542b270-a175-4b43-9295-8096d4eb4092 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:11:58.564341 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6542b270-a175-4b43-9295-8096d4eb4092 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:11:58.633963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6542b270-a175-4b43-9295-8096d4eb4092 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:11:58.635389 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6542b270-a175-4b43-9295-8096d4eb4092 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:11:58.653355 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1276/2537] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:11:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:12:08.484798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=33 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:12:08.650832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=36 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:12:30.591526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70aa478-3f3e-4376-8e1c-2749718c5253 service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:12:30.592899 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-b70aa478-3f3e-4376-8e1c-2749718c5253 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:12:30.659804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70aa478-3f3e-4376-8e1c-2749718c5253 service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:12:30.665134 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6146fb7-6898-494e-aa44-76b2dad5f649 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350f72f740f749f68175c1c4e8ae9d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:12:30.665766 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6146fb7-6898-494e-aa44-76b2dad5f649 service placement] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:12:30.666044 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6146fb7-6898-494e-aa44-76b2dad5f649 service placement] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:12:30.666310 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6146fb7-6898-494e-aa44-76b2dad5f649 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:12:30.668041 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c6146fb7-6898-494e-aa44-76b2dad5f649 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:12:30.679648 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6146fb7-6898-494e-aa44-76b2dad5f649 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:12:30.681188 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6146fb7-6898-494e-aa44-76b2dad5f649 service placement] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:12:30.690679 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1262/2538] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:12:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:12:40.686690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=32 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:13:58.871036 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542b270-a175-4b43-9295-8096d4eb4092 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:13:58.879992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542b270-a175-4b43-9295-8096d4eb4092 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:13:58.883847 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57011f6f-d5fe-4c74-abef-3f85fdd0c4cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:13:58.884463 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57011f6f-d5fe-4c74-abef-3f85fdd0c4cf service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:13:58.884728 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57011f6f-d5fe-4c74-abef-3f85fdd0c4cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:13:58.885001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57011f6f-d5fe-4c74-abef-3f85fdd0c4cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:13:58.886791 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-57011f6f-d5fe-4c74-abef-3f85fdd0c4cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:13:58.945251 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57011f6f-d5fe-4c74-abef-3f85fdd0c4cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:13:58.947444 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57011f6f-d5fe-4c74-abef-3f85fdd0c4cf service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:13:58.956153 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1277/2539] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:13:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:08.952179 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=28 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:10.770054 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6146fb7-6898-494e-aa44-76b2dad5f649 service placement] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:10.868864 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6146fb7-6898-494e-aa44-76b2dad5f649 service placement] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:10.873243 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12dfca1-2452-4864-99a2-26f074caa0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:10.874082 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12dfca1-2452-4864-99a2-26f074caa0a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:10.874362 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12dfca1-2452-4864-99a2-26f074caa0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:10.874707 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12dfca1-2452-4864-99a2-26f074caa0a4 admin admin] PATH_INFO: `/v3/users/af595b52687f4027b505adf49e1ebffc` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:10.876042 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12dfca1-2452-4864-99a2-26f074caa0a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af595b52687f4027b505adf49e1ebffc)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:10.878562 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12dfca1-2452-4864-99a2-26f074caa0a4 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:10.930199 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d12dfca1-2452-4864-99a2-26f074caa0a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af595b52687f4027b505adf49e1ebffc'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:10.930976 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d12dfca1-2452-4864-99a2-26f074caa0a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af595b52687f4027b505adf49e1ebffc'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:10.945796 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d12dfca1-2452-4864-99a2-26f074caa0a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af595b52687f4027b505adf49e1ebffc'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:10.953345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d12dfca1-2452-4864-99a2-26f074caa0a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af595b52687f4027b505adf49e1ebffc'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:10.953991 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d12dfca1-2452-4864-99a2-26f074caa0a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af595b52687f4027b505adf49e1ebffc'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:10.968023 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1263/2540] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:14:10 2022] DELETE /identity/v3/users/af595b52687f4027b505adf49e1ebffc => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:10.972541 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c991a-2c2a-46fc-bb06-063625daebf7 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:10.973041 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c991a-2c2a-46fc-bb06-063625daebf7 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:10.973169 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c991a-2c2a-46fc-bb06-063625daebf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:11.005677 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7d4c991a-2c2a-46fc-bb06-063625daebf7 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:11.006954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7d4c991a-2c2a-46fc-bb06-063625daebf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:11.071980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1278/2541] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:11.082884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12dfca1-2452-4864-99a2-26f074caa0a4 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:11.084440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d12dfca1-2452-4864-99a2-26f074caa0a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:11.148501 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12dfca1-2452-4864-99a2-26f074caa0a4 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:11.152645 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d176b-e6e9-43d9-b11b-6dd41cd75022 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:11.153263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d176b-e6e9-43d9-b11b-6dd41cd75022 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:11.153564 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d176b-e6e9-43d9-b11b-6dd41cd75022 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:11.153843 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d176b-e6e9-43d9-b11b-6dd41cd75022 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:11.163733 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d176b-e6e9-43d9-b11b-6dd41cd75022 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:11.165658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d176b-e6e9-43d9-b11b-6dd41cd75022 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:11.176236 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1264/2542] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:11.483441 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c991a-2c2a-46fc-bb06-063625daebf7 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:11.484905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7d4c991a-2c2a-46fc-bb06-063625daebf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:11.497298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c991a-2c2a-46fc-bb06-063625daebf7 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:11.502319 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d2d331-1b30-4798-98a8-122e2ae7fe88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:11.503189 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2d331-1b30-4798-98a8-122e2ae7fe88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:11.503330 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2d331-1b30-4798-98a8-122e2ae7fe88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:11.503518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2d331-1b30-4798-98a8-122e2ae7fe88 admin admin] PATH_INFO: `/v3/projects/922768dc24754a8a8ea7c774fe2ccb45` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:11.504861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d2d331-1b30-4798-98a8-122e2ae7fe88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=922768dc24754a8a8ea7c774fe2ccb45)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:11.506947 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d2d331-1b30-4798-98a8-122e2ae7fe88 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:11.573483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-98d2d331-1b30-4798-98a8-122e2ae7fe88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '922768dc24754a8a8ea7c774fe2ccb45', 'request_id': 'req-98d2d331-1b30-4798-98a8-122e2ae7fe88'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:11.582228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-98d2d331-1b30-4798-98a8-122e2ae7fe88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '922768dc24754a8a8ea7c774fe2ccb45', 'request_id': 'req-98d2d331-1b30-4798-98a8-122e2ae7fe88'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:11.588516 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-98d2d331-1b30-4798-98a8-122e2ae7fe88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '922768dc24754a8a8ea7c774fe2ccb45', 'request_id': 'req-98d2d331-1b30-4798-98a8-122e2ae7fe88'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:11.600664 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-98d2d331-1b30-4798-98a8-122e2ae7fe88 admin admin] The token cache is being invalidate because project 922768dc24754a8a8ea7c774fe2ccb45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:14:11.601113 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-98d2d331-1b30-4798-98a8-122e2ae7fe88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:11.602862 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-98d2d331-1b30-4798-98a8-122e2ae7fe88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:11.604014 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1279/2543] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:14:11 2022] DELETE /identity/v3/projects/922768dc24754a8a8ea7c774fe2ccb45 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:11.608030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d176b-e6e9-43d9-b11b-6dd41cd75022 service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:11.614015 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:11.618530 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d176b-e6e9-43d9-b11b-6dd41cd75022 service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:11.678529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ba1a3-3e35-4d3d-aad9-05e8f51311b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:11.679084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ba1a3-3e35-4d3d-aad9-05e8f51311b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:11.679378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ba1a3-3e35-4d3d-aad9-05e8f51311b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:11.679633 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ba1a3-3e35-4d3d-aad9-05e8f51311b5 admin admin] PATH_INFO: `/v3/users/fb61e0e45ea44a2582d0bdab866c9bb3` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:11.681372 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ba1a3-3e35-4d3d-aad9-05e8f51311b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb61e0e45ea44a2582d0bdab866c9bb3)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:11.683292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ba1a3-3e35-4d3d-aad9-05e8f51311b5 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:11.754188 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-069ba1a3-3e35-4d3d-aad9-05e8f51311b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb61e0e45ea44a2582d0bdab866c9bb3'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:11.755220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-069ba1a3-3e35-4d3d-aad9-05e8f51311b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb61e0e45ea44a2582d0bdab866c9bb3'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:11.772755 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-069ba1a3-3e35-4d3d-aad9-05e8f51311b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb61e0e45ea44a2582d0bdab866c9bb3'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:11.794119 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-069ba1a3-3e35-4d3d-aad9-05e8f51311b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb61e0e45ea44a2582d0bdab866c9bb3'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:11.794893 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-069ba1a3-3e35-4d3d-aad9-05e8f51311b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb61e0e45ea44a2582d0bdab866c9bb3'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:11.826683 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1265/2544] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:14:11 2022] DELETE /identity/v3/users/fb61e0e45ea44a2582d0bdab866c9bb3 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:11.848466 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d2d331-1b30-4798-98a8-122e2ae7fe88 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:11.850152 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-98d2d331-1b30-4798-98a8-122e2ae7fe88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:11.937697 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d2d331-1b30-4798-98a8-122e2ae7fe88 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:11.983454 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:12.032824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b0fdc-45d1-4159-abff-c2b889bbc9e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:12.033452 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b0fdc-45d1-4159-abff-c2b889bbc9e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:12.034298 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b0fdc-45d1-4159-abff-c2b889bbc9e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:12.034589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b0fdc-45d1-4159-abff-c2b889bbc9e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:12.036583 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-779b0fdc-45d1-4159-abff-c2b889bbc9e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:12.041757 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ba1a3-3e35-4d3d-aad9-05e8f51311b5 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:12.043534 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-069ba1a3-3e35-4d3d-aad9-05e8f51311b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:12.065579 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ba1a3-3e35-4d3d-aad9-05e8f51311b5 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:12.070945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de2ccc-b0d9-42ae-9480-e8e358409c71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:12.072898 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de2ccc-b0d9-42ae-9480-e8e358409c71 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:12.073220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de2ccc-b0d9-42ae-9480-e8e358409c71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:12.073688 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de2ccc-b0d9-42ae-9480-e8e358409c71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:12.076116 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-30de2ccc-b0d9-42ae-9480-e8e358409c71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:12.140155 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b0fdc-45d1-4159-abff-c2b889bbc9e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:12.141807 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b0fdc-45d1-4159-abff-c2b889bbc9e6 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:12.153992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1280/2545] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:12.154991 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b0fdc-45d1-4159-abff-c2b889bbc9e6 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:12.167456 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b0fdc-45d1-4159-abff-c2b889bbc9e6 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:12.175167 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2292c-b3a2-4b25-99a3-27ab5b099fd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:12.175809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2292c-b3a2-4b25-99a3-27ab5b099fd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:12.176092 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2292c-b3a2-4b25-99a3-27ab5b099fd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:12.176422 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2292c-b3a2-4b25-99a3-27ab5b099fd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:12.191915 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-75a2292c-b3a2-4b25-99a3-27ab5b099fd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:12.211788 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:12.216014 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2292c-b3a2-4b25-99a3-27ab5b099fd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:12.221597 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2292c-b3a2-4b25-99a3-27ab5b099fd7 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:12.231325 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de2ccc-b0d9-42ae-9480-e8e358409c71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:12.234009 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de2ccc-b0d9-42ae-9480-e8e358409c71 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:12.235929 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1281/2546] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:12.237207 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2292c-b3a2-4b25-99a3-27ab5b099fd7 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:12.241071 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-75a2292c-b3a2-4b25-99a3-27ab5b099fd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:12.263192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1266/2547] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:12.276315 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2292c-b3a2-4b25-99a3-27ab5b099fd7 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:12.283020 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198366b-ee7f-48ac-91ae-8aa51a882617 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:12.283761 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198366b-ee7f-48ac-91ae-8aa51a882617 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:12.284119 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198366b-ee7f-48ac-91ae-8aa51a882617 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:12.284667 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198366b-ee7f-48ac-91ae-8aa51a882617 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:12.286761 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6198366b-ee7f-48ac-91ae-8aa51a882617 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:12.304612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de2ccc-b0d9-42ae-9480-e8e358409c71 service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:12.307297 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-30de2ccc-b0d9-42ae-9480-e8e358409c71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:12.310236 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198366b-ee7f-48ac-91ae-8aa51a882617 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:12.313490 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198366b-ee7f-48ac-91ae-8aa51a882617 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:12.322974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de2ccc-b0d9-42ae-9480-e8e358409c71 service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:12.334734 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1282/2548] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:12.335002 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1560a72-dbbe-4be7-a440-5df6b51aff5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:12.335485 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1560a72-dbbe-4be7-a440-5df6b51aff5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:12.335771 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1560a72-dbbe-4be7-a440-5df6b51aff5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:12.337501 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1560a72-dbbe-4be7-a440-5df6b51aff5f admin admin] PATH_INFO: `/v3/projects/be441fa5c8274b4ab7baf52de8a1e644` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:12.349319 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1560a72-dbbe-4be7-a440-5df6b51aff5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=be441fa5c8274b4ab7baf52de8a1e644)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:12.354295 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1560a72-dbbe-4be7-a440-5df6b51aff5f admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:12.428248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a1560a72-dbbe-4be7-a440-5df6b51aff5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be441fa5c8274b4ab7baf52de8a1e644', 'request_id': 'req-a1560a72-dbbe-4be7-a440-5df6b51aff5f'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:12.436592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a1560a72-dbbe-4be7-a440-5df6b51aff5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be441fa5c8274b4ab7baf52de8a1e644', 'request_id': 'req-a1560a72-dbbe-4be7-a440-5df6b51aff5f'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:12.442676 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a1560a72-dbbe-4be7-a440-5df6b51aff5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be441fa5c8274b4ab7baf52de8a1e644', 'request_id': 'req-a1560a72-dbbe-4be7-a440-5df6b51aff5f'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:12.454197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a1560a72-dbbe-4be7-a440-5df6b51aff5f admin admin] The token cache is being invalidate because project be441fa5c8274b4ab7baf52de8a1e644 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:14:12.454582 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a1560a72-dbbe-4be7-a440-5df6b51aff5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:12.455288 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a1560a72-dbbe-4be7-a440-5df6b51aff5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:12.456846 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1267/2549] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:14:12 2022] DELETE /identity/v3/projects/be441fa5c8274b4ab7baf52de8a1e644 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 05:14:12.462383 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8abc3-d66d-46ea-b92e-981216c5d108 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:12.462699 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8abc3-d66d-46ea-b92e-981216c5d108 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:12.462980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8abc3-d66d-46ea-b92e-981216c5d108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:12.484171 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-63e8abc3-d66d-46ea-b92e-981216c5d108 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:12.485570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-63e8abc3-d66d-46ea-b92e-981216c5d108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:12.547887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1283/2550] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:12.556163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6d83a-a47a-4593-b287-94d255b5f64e None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:12.556442 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6d83a-a47a-4593-b287-94d255b5f64e None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:12.556715 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6d83a-a47a-4593-b287-94d255b5f64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:12.578179 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-3cc6d83a-a47a-4593-b287-94d255b5f64e None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:12.579530 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-3cc6d83a-a47a-4593-b287-94d255b5f64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:12.597117 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1268/2551] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 05:14:12.603098 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8abc3-d66d-46ea-b92e-981216c5d108 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:12.617557 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8abc3-d66d-46ea-b92e-981216c5d108 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:12.624324 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1436d0-16b1-4f7d-b383-9601ff5be8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:12.625570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1436d0-16b1-4f7d-b383-9601ff5be8be admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:12.625570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1436d0-16b1-4f7d-b383-9601ff5be8be admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:12.626218 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1436d0-16b1-4f7d-b383-9601ff5be8be admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:12.627317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1436d0-16b1-4f7d-b383-9601ff5be8be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 05:14:12.627751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1436d0-16b1-4f7d-b383-9601ff5be8be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:12.631271 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1436d0-16b1-4f7d-b383-9601ff5be8be admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:12.648389 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1284/2552] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 05:14:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 05:14:12.653228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6d83a-a47a-4593-b287-94d255b5f64e None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:12.665889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6d83a-a47a-4593-b287-94d255b5f64e None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:12.681322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141e6ac-b798-41e0-960d-d65b59b465db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:12.683062 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141e6ac-b798-41e0-960d-d65b59b465db admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:12.683531 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141e6ac-b798-41e0-960d-d65b59b465db admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:12.683969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141e6ac-b798-41e0-960d-d65b59b465db admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:12.685265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141e6ac-b798-41e0-960d-d65b59b465db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:12.688842 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141e6ac-b798-41e0-960d-d65b59b465db admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:12.744657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1269/2553] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:14:12 2022] POST /identity/v3/projects => generated 375 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:14:12.749034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1436d0-16b1-4f7d-b383-9601ff5be8be admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:12.762481 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1436d0-16b1-4f7d-b383-9601ff5be8be admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:12.805645 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:12.836732 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede5d47-a6d4-43d5-9def-9265fdfd77e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:12.837428 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede5d47-a6d4-43d5-9def-9265fdfd77e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:12.837748 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede5d47-a6d4-43d5-9def-9265fdfd77e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:12.838095 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede5d47-a6d4-43d5-9def-9265fdfd77e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:12.838845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede5d47-a6d4-43d5-9def-9265fdfd77e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:12.841814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede5d47-a6d4-43d5-9def-9265fdfd77e6 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:12.866500 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1285/2554] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:14:12 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:14:12.871309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141e6ac-b798-41e0-960d-d65b59b465db admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:12.882345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141e6ac-b798-41e0-960d-d65b59b465db admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:12.888738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c0977-81c7-4f55-aa56-5a6a1cf79ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:12.889193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c0977-81c7-4f55-aa56-5a6a1cf79ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:12.889565 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c0977-81c7-4f55-aa56-5a6a1cf79ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:12.890560 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c0977-81c7-4f55-aa56-5a6a1cf79ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:12.890995 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c0977-81c7-4f55-aa56-5a6a1cf79ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:12.892695 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c0977-81c7-4f55-aa56-5a6a1cf79ef8 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:12.905622 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1270/2555] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 05:14:12.909292 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede5d47-a6d4-43d5-9def-9265fdfd77e6 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:12.926002 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede5d47-a6d4-43d5-9def-9265fdfd77e6 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:12.931796 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95f6e9-3258-443c-b96f-cb4c367067d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:12.932529 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95f6e9-3258-443c-b96f-cb4c367067d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:12.932908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95f6e9-3258-443c-b96f-cb4c367067d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:12.933281 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95f6e9-3258-443c-b96f-cb4c367067d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:12.934115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95f6e9-3258-443c-b96f-cb4c367067d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:12.935821 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95f6e9-3258-443c-b96f-cb4c367067d7 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:12.948326 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1286/2556] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:14:12.952585 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c0977-81c7-4f55-aa56-5a6a1cf79ef8 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:12.967676 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c0977-81c7-4f55-aa56-5a6a1cf79ef8 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:12.973312 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9dde3f-6882-47e7-af4c-ecbfc44e2664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:12.974250 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9dde3f-6882-47e7-af4c-ecbfc44e2664 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:12.974831 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9dde3f-6882-47e7-af4c-ecbfc44e2664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:12.975279 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9dde3f-6882-47e7-af4c-ecbfc44e2664 admin admin] PATH_INFO: `/v3/projects/ee56079a41714a81858f12ec6ef52468/users/ea11310ddd8249cc93226dd25abaed35/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:12.992025 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9dde3f-6882-47e7-af4c-ecbfc44e2664 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee56079a41714a81858f12ec6ef52468, user_id=ea11310ddd8249cc93226dd25abaed35, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:12.996054 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9dde3f-6882-47e7-af4c-ecbfc44e2664 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:13.011526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1271/2557] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:14:12 2022] PUT /identity/v3/projects/ee56079a41714a81858f12ec6ef52468/users/ea11310ddd8249cc93226dd25abaed35/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 05:14:13.023173 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38ee60-ec62-41e0-9e16-a135584c02ee None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:13.023960 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38ee60-ec62-41e0-9e16-a135584c02ee None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:13.024314 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38ee60-ec62-41e0-9e16-a135584c02ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:13.054249 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ae38ee60-ec62-41e0-9e16-a135584c02ee None None] MFA Rules not processed for user `ea11310ddd8249cc93226dd25abaed35`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:13.056238 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ae38ee60-ec62-41e0-9e16-a135584c02ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:13.131146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1287/2558] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:13.144572 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9dde3f-6882-47e7-af4c-ecbfc44e2664 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:13.147409 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-fc9dde3f-6882-47e7-af4c-ecbfc44e2664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:13.225210 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38ee60-ec62-41e0-9e16-a135584c02ee None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:13.228271 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ae38ee60-ec62-41e0-9e16-a135584c02ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:13.246650 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9dde3f-6882-47e7-af4c-ecbfc44e2664 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:13.251933 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46305331-6b4f-4777-be83-6f1d8b9d69ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:13.252671 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46305331-6b4f-4777-be83-6f1d8b9d69ce service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:13.253017 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46305331-6b4f-4777-be83-6f1d8b9d69ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:13.253368 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46305331-6b4f-4777-be83-6f1d8b9d69ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:13.267309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46305331-6b4f-4777-be83-6f1d8b9d69ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:13.269171 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46305331-6b4f-4777-be83-6f1d8b9d69ce service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:13.286727 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1272/2559] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:13.349245 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38ee60-ec62-41e0-9e16-a135584c02ee None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:13.356122 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a4301-83be-4553-9c9f-b6ac587c3873 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350f72f740f749f68175c1c4e8ae9d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:13.357379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a4301-83be-4553-9c9f-b6ac587c3873 service placement] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:13.357711 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a4301-83be-4553-9c9f-b6ac587c3873 service placement] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:13.358066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a4301-83be-4553-9c9f-b6ac587c3873 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:13.360061 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-be9a4301-83be-4553-9c9f-b6ac587c3873 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:13.379871 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a4301-83be-4553-9c9f-b6ac587c3873 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:13.381473 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a4301-83be-4553-9c9f-b6ac587c3873 service placement] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:13.400461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1288/2560] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:13.702829 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46305331-6b4f-4777-be83-6f1d8b9d69ce service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:13.704577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-46305331-6b4f-4777-be83-6f1d8b9d69ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:13.713064 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:13.827385 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a4301-83be-4553-9c9f-b6ac587c3873 service placement] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:13.829014 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-be9a4301-83be-4553-9c9f-b6ac587c3873 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:13.843661 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46305331-6b4f-4777-be83-6f1d8b9d69ce service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:13.850562 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49cffa-e01d-47aa-aa2a-a8ee59f2c044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:13.860488 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49cffa-e01d-47aa-aa2a-a8ee59f2c044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:13.861082 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49cffa-e01d-47aa-aa2a-a8ee59f2c044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:13.861617 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49cffa-e01d-47aa-aa2a-a8ee59f2c044 admin admin] PATH_INFO: `/v3/users/d604afd3c4234473ac1f2fcff2501743` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:13.863361 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49cffa-e01d-47aa-aa2a-a8ee59f2c044 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d604afd3c4234473ac1f2fcff2501743)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:13.866006 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49cffa-e01d-47aa-aa2a-a8ee59f2c044 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:13.890759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:13.916480 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a4301-83be-4553-9c9f-b6ac587c3873 service placement] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:13.920961 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb153d-3000-446c-af3e-b95c3949cc98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:13.922101 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb153d-3000-446c-af3e-b95c3949cc98 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:13.922101 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb153d-3000-446c-af3e-b95c3949cc98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:13.922524 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb153d-3000-446c-af3e-b95c3949cc98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:13.924520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9deb153d-3000-446c-af3e-b95c3949cc98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:13.952758 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-0f49cffa-e01d-47aa-aa2a-a8ee59f2c044 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd604afd3c4234473ac1f2fcff2501743'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:13.954324 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-0f49cffa-e01d-47aa-aa2a-a8ee59f2c044 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd604afd3c4234473ac1f2fcff2501743'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:13.972636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-0f49cffa-e01d-47aa-aa2a-a8ee59f2c044 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd604afd3c4234473ac1f2fcff2501743'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:13.997347 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-0f49cffa-e01d-47aa-aa2a-a8ee59f2c044 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd604afd3c4234473ac1f2fcff2501743'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:14.000200 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-0f49cffa-e01d-47aa-aa2a-a8ee59f2c044 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd604afd3c4234473ac1f2fcff2501743'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:14.019269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1273/2561] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:14:13 2022] DELETE /identity/v3/users/d604afd3c4234473ac1f2fcff2501743 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:14.040370 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb153d-3000-446c-af3e-b95c3949cc98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:14.042523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb153d-3000-446c-af3e-b95c3949cc98 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:14.056271 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1289/2562] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 05:14:14.256047 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49cffa-e01d-47aa-aa2a-a8ee59f2c044 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:14.257729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-0f49cffa-e01d-47aa-aa2a-a8ee59f2c044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:14.356944 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49cffa-e01d-47aa-aa2a-a8ee59f2c044 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:14.369156 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90439f92-8e48-4dde-9c88-c83d9fb54916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:14.370075 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90439f92-8e48-4dde-9c88-c83d9fb54916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:14.370681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90439f92-8e48-4dde-9c88-c83d9fb54916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:14.370681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90439f92-8e48-4dde-9c88-c83d9fb54916 admin admin] PATH_INFO: `/v3/projects/58d8f4b47019401ab6e953bbbbd801fd` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:14.372228 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90439f92-8e48-4dde-9c88-c83d9fb54916 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58d8f4b47019401ab6e953bbbbd801fd)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:14.374463 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90439f92-8e48-4dde-9c88-c83d9fb54916 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:14.432110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:14.461286 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-90439f92-8e48-4dde-9c88-c83d9fb54916 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58d8f4b47019401ab6e953bbbbd801fd', 'request_id': 'req-90439f92-8e48-4dde-9c88-c83d9fb54916'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:14.471292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-90439f92-8e48-4dde-9c88-c83d9fb54916 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58d8f4b47019401ab6e953bbbbd801fd', 'request_id': 'req-90439f92-8e48-4dde-9c88-c83d9fb54916'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:14.481363 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-90439f92-8e48-4dde-9c88-c83d9fb54916 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58d8f4b47019401ab6e953bbbbd801fd', 'request_id': 'req-90439f92-8e48-4dde-9c88-c83d9fb54916'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:14.496418 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-90439f92-8e48-4dde-9c88-c83d9fb54916 admin admin] The token cache is being invalidate because project 58d8f4b47019401ab6e953bbbbd801fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:14:14.497338 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-90439f92-8e48-4dde-9c88-c83d9fb54916 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:14.498842 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-90439f92-8e48-4dde-9c88-c83d9fb54916 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:14.501919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1274/2563] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:14:14 2022] DELETE /identity/v3/projects/58d8f4b47019401ab6e953bbbbd801fd => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:14.507671 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece71c41-20aa-4ee5-95c6-8cc965580d0c None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:14.508513 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece71c41-20aa-4ee5-95c6-8cc965580d0c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:14.508960 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece71c41-20aa-4ee5-95c6-8cc965580d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:14.532764 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ece71c41-20aa-4ee5-95c6-8cc965580d0c None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:14.534021 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ece71c41-20aa-4ee5-95c6-8cc965580d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:14.605642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1290/2564] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:14.614898 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8f714-2b7c-4b39-969f-1df195cd094d None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:14.614898 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8f714-2b7c-4b39-969f-1df195cd094d None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:14.615079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8f714-2b7c-4b39-969f-1df195cd094d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:14.637206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-fdd8f714-2b7c-4b39-969f-1df195cd094d None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:14.638549 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-fdd8f714-2b7c-4b39-969f-1df195cd094d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:14.646915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1275/2565] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 05:14:14.651813 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece71c41-20aa-4ee5-95c6-8cc965580d0c None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:14.661864 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece71c41-20aa-4ee5-95c6-8cc965580d0c None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:14.667505 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac9f74-3b15-4604-884b-dbcdf754aeaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:14.668118 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac9f74-3b15-4604-884b-dbcdf754aeaa admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:14.668679 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac9f74-3b15-4604-884b-dbcdf754aeaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:14.668973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac9f74-3b15-4604-884b-dbcdf754aeaa admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:14.669720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac9f74-3b15-4604-884b-dbcdf754aeaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 05:14:14.670031 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac9f74-3b15-4604-884b-dbcdf754aeaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:14.671684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac9f74-3b15-4604-884b-dbcdf754aeaa admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:14.691646 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1291/2566] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 05:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 05:14:14.692826 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8f714-2b7c-4b39-969f-1df195cd094d None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:14.703838 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8f714-2b7c-4b39-969f-1df195cd094d None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:14.711691 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18c9b2-a6bc-4526-a3c8-e0fc5a992499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:14.712653 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18c9b2-a6bc-4526-a3c8-e0fc5a992499 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:14.713123 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18c9b2-a6bc-4526-a3c8-e0fc5a992499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:14.713402 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18c9b2-a6bc-4526-a3c8-e0fc5a992499 admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:14.714996 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18c9b2-a6bc-4526-a3c8-e0fc5a992499 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:14.717662 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18c9b2-a6bc-4526-a3c8-e0fc5a992499 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:14.750329 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1276/2567] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:14:14 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 05:14:14.754215 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac9f74-3b15-4604-884b-dbcdf754aeaa admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:14.770645 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac9f74-3b15-4604-884b-dbcdf754aeaa admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:14.865524 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d47e6e-1573-483c-bb5f-48f649ba4242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:14.866483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d47e6e-1573-483c-bb5f-48f649ba4242 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:14.866770 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d47e6e-1573-483c-bb5f-48f649ba4242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:14.867048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d47e6e-1573-483c-bb5f-48f649ba4242 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:14.870488 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d47e6e-1573-483c-bb5f-48f649ba4242 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:14.872461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d47e6e-1573-483c-bb5f-48f649ba4242 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:14.887907 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:14.918881 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1292/2568] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:14:14 2022] POST /identity/v3/users => generated 290 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 05:14:14.919793 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18c9b2-a6bc-4526-a3c8-e0fc5a992499 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:14.932565 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18c9b2-a6bc-4526-a3c8-e0fc5a992499 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:14.938665 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4751b96-c50c-45fe-bd78-7f130133a7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:14.939775 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4751b96-c50c-45fe-bd78-7f130133a7da admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:14.940293 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4751b96-c50c-45fe-bd78-7f130133a7da admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:14.940786 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4751b96-c50c-45fe-bd78-7f130133a7da admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:14.941952 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4751b96-c50c-45fe-bd78-7f130133a7da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:14.944530 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4751b96-c50c-45fe-bd78-7f130133a7da admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:14.959339 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1277/2569] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 05:14:14.964393 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d47e6e-1573-483c-bb5f-48f649ba4242 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:14.975884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d47e6e-1573-483c-bb5f-48f649ba4242 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:14.983008 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466064ca-bb1c-4952-916f-7d1225b0cc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:14.983694 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466064ca-bb1c-4952-916f-7d1225b0cc0c admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:14.984082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466064ca-bb1c-4952-916f-7d1225b0cc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:14.984674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466064ca-bb1c-4952-916f-7d1225b0cc0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:14.985686 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466064ca-bb1c-4952-916f-7d1225b0cc0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:14.987641 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466064ca-bb1c-4952-916f-7d1225b0cc0c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:15.005334 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1293/2570] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 05:14:15.010316 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4751b96-c50c-45fe-bd78-7f130133a7da admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:15.029512 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4751b96-c50c-45fe-bd78-7f130133a7da admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:15.035499 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf4a6c-7ef9-41e1-8e3c-ff9803d0598a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:15.036111 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf4a6c-7ef9-41e1-8e3c-ff9803d0598a admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:15.036505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf4a6c-7ef9-41e1-8e3c-ff9803d0598a admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:15.036807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf4a6c-7ef9-41e1-8e3c-ff9803d0598a admin admin] PATH_INFO: `/v3/projects/731637c0b96a4f1f8d9edb1af69e98cb/users/30c0ff27c5a84ebd8615fb4608fc3882/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:15.061231 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf4a6c-7ef9-41e1-8e3c-ff9803d0598a admin admin] RBAC: Authorizing `identity:create_grant(project_id=731637c0b96a4f1f8d9edb1af69e98cb, user_id=30c0ff27c5a84ebd8615fb4608fc3882, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:15.063983 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf4a6c-7ef9-41e1-8e3c-ff9803d0598a admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:15.080124 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1278/2571] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:14:15 2022] PUT /identity/v3/projects/731637c0b96a4f1f8d9edb1af69e98cb/users/30c0ff27c5a84ebd8615fb4608fc3882/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:15.086164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd9fb1-5c2e-4f71-b066-1b755e34d15a None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:15.087148 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd9fb1-5c2e-4f71-b066-1b755e34d15a None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:15.087428 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd9fb1-5c2e-4f71-b066-1b755e34d15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:15.108394 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-a0fd9fb1-5c2e-4f71-b066-1b755e34d15a None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:15.109683 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a0fd9fb1-5c2e-4f71-b066-1b755e34d15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:15.181582 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1294/2572] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:15.190730 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf4a6c-7ef9-41e1-8e3c-ff9803d0598a admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:15.192520 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-bfbf4a6c-7ef9-41e1-8e3c-ff9803d0598a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:15.256946 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf4a6c-7ef9-41e1-8e3c-ff9803d0598a admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:15.262106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de0b07-d27e-4d39-ad72-1f764c9556f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:15.262832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de0b07-d27e-4d39-ad72-1f764c9556f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:15.263116 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de0b07-d27e-4d39-ad72-1f764c9556f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:15.263386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de0b07-d27e-4d39-ad72-1f764c9556f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:15.274369 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de0b07-d27e-4d39-ad72-1f764c9556f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:15.276024 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de0b07-d27e-4d39-ad72-1f764c9556f8 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:15.288694 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1279/2573] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 05:14:16.369909 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd9fb1-5c2e-4f71-b066-1b755e34d15a None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:16.371675 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a0fd9fb1-5c2e-4f71-b066-1b755e34d15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:16.393518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd9fb1-5c2e-4f71-b066-1b755e34d15a None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:16.400503 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aca47a-3fd2-4191-b953-72ac0fec2a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:16.401609 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aca47a-3fd2-4191-b953-72ac0fec2a08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:16.402022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aca47a-3fd2-4191-b953-72ac0fec2a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:16.402371 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aca47a-3fd2-4191-b953-72ac0fec2a08 admin admin] PATH_INFO: `/v3/users/49a7ce9a27734dc0a187d1b217206785` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:16.403968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aca47a-3fd2-4191-b953-72ac0fec2a08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49a7ce9a27734dc0a187d1b217206785)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:16.406138 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aca47a-3fd2-4191-b953-72ac0fec2a08 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:16.492866 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-60aca47a-3fd2-4191-b953-72ac0fec2a08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49a7ce9a27734dc0a187d1b217206785'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:16.502374 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-60aca47a-3fd2-4191-b953-72ac0fec2a08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49a7ce9a27734dc0a187d1b217206785'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:16.503676 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-60aca47a-3fd2-4191-b953-72ac0fec2a08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49a7ce9a27734dc0a187d1b217206785'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:16.527304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-60aca47a-3fd2-4191-b953-72ac0fec2a08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49a7ce9a27734dc0a187d1b217206785'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:16.528290 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-60aca47a-3fd2-4191-b953-72ac0fec2a08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49a7ce9a27734dc0a187d1b217206785'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:16.546919 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1295/2574] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:14:16 2022] DELETE /identity/v3/users/49a7ce9a27734dc0a187d1b217206785 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:16.983269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de0b07-d27e-4d39-ad72-1f764c9556f8 service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:16.995489 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de0b07-d27e-4d39-ad72-1f764c9556f8 service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:17.058432 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:17.073768 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d6bba9-4d36-4eca-80f1-41060c548ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:17.075073 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6bba9-4d36-4eca-80f1-41060c548ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:17.075526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6bba9-4d36-4eca-80f1-41060c548ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:17.075987 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6bba9-4d36-4eca-80f1-41060c548ebe admin admin] PATH_INFO: `/v3/projects/9124d19d48f44fe2a904a715e5db1223` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:17.077207 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d6bba9-4d36-4eca-80f1-41060c548ebe admin admin] RBAC: Authorizing `identity:delete_project(project_id=9124d19d48f44fe2a904a715e5db1223)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:17.079149 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d6bba9-4d36-4eca-80f1-41060c548ebe admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:17.158900 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-46d6bba9-4d36-4eca-80f1-41060c548ebe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9124d19d48f44fe2a904a715e5db1223', 'request_id': 'req-46d6bba9-4d36-4eca-80f1-41060c548ebe'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:17.177042 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-46d6bba9-4d36-4eca-80f1-41060c548ebe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9124d19d48f44fe2a904a715e5db1223', 'request_id': 'req-46d6bba9-4d36-4eca-80f1-41060c548ebe'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:17.186099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-46d6bba9-4d36-4eca-80f1-41060c548ebe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9124d19d48f44fe2a904a715e5db1223', 'request_id': 'req-46d6bba9-4d36-4eca-80f1-41060c548ebe'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:17.210672 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-46d6bba9-4d36-4eca-80f1-41060c548ebe admin admin] The token cache is being invalidate because project 9124d19d48f44fe2a904a715e5db1223 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:14:17.211658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-46d6bba9-4d36-4eca-80f1-41060c548ebe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:17.212292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-46d6bba9-4d36-4eca-80f1-41060c548ebe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:17.214305 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1280/2575] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:14:16 2022] DELETE /identity/v3/projects/9124d19d48f44fe2a904a715e5db1223 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:17.219884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82df27-b4cd-4c72-821d-6f1d8ff7c28e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:17.220177 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82df27-b4cd-4c72-821d-6f1d8ff7c28e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:17.220459 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82df27-b4cd-4c72-821d-6f1d8ff7c28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:17.242944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7a82df27-b4cd-4c72-821d-6f1d8ff7c28e None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:17.244299 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7a82df27-b4cd-4c72-821d-6f1d8ff7c28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:17.302215 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:17.316222 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1296/2576] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:17.326272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48caa4c7-5ec2-447c-b65e-ade8caaf3ab1 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:17.326553 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48caa4c7-5ec2-447c-b65e-ade8caaf3ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:17.326824 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48caa4c7-5ec2-447c-b65e-ade8caaf3ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:17.356066 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-48caa4c7-5ec2-447c-b65e-ade8caaf3ab1 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:17.356066 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-48caa4c7-5ec2-447c-b65e-ade8caaf3ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:17.369287 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1281/2577] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:17.374191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82df27-b4cd-4c72-821d-6f1d8ff7c28e None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:17.394581 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82df27-b4cd-4c72-821d-6f1d8ff7c28e None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:17.400872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb59313-d458-4410-a0bc-660fc999ae8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:17.401503 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb59313-d458-4410-a0bc-660fc999ae8f admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:17.401833 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb59313-d458-4410-a0bc-660fc999ae8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:17.402109 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb59313-d458-4410-a0bc-660fc999ae8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:17.402827 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb59313-d458-4410-a0bc-660fc999ae8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 05:14:17.403311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb59313-d458-4410-a0bc-660fc999ae8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:17.404950 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb59313-d458-4410-a0bc-660fc999ae8f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:17.418908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1297/2578] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 05:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 05:14:17.424360 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48caa4c7-5ec2-447c-b65e-ade8caaf3ab1 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:17.443857 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48caa4c7-5ec2-447c-b65e-ade8caaf3ab1 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:17.452102 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340fa345-f084-4863-8ce5-3008fe324a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:17.452743 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fa345-f084-4863-8ce5-3008fe324a2a admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:17.453024 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fa345-f084-4863-8ce5-3008fe324a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:17.453296 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fa345-f084-4863-8ce5-3008fe324a2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:17.454068 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340fa345-f084-4863-8ce5-3008fe324a2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:17.456001 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340fa345-f084-4863-8ce5-3008fe324a2a admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:17.496322 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1282/2579] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:14:17 2022] POST /identity/v3/projects => generated 369 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:14:17.502841 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb59313-d458-4410-a0bc-660fc999ae8f admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:17.514028 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb59313-d458-4410-a0bc-660fc999ae8f admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:17.582483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2b490-cf2c-4b88-b389-21dcf314749b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:17.583098 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2b490-cf2c-4b88-b389-21dcf314749b admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:17.583380 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2b490-cf2c-4b88-b389-21dcf314749b admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:17.583651 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2b490-cf2c-4b88-b389-21dcf314749b admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:17.584318 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2b490-cf2c-4b88-b389-21dcf314749b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:17.586360 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2b490-cf2c-4b88-b389-21dcf314749b admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:17.608398 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1298/2580] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:14:17 2022] POST /identity/v3/users => generated 285 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:14:17.612727 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340fa345-f084-4863-8ce5-3008fe324a2a admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:17.624404 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340fa345-f084-4863-8ce5-3008fe324a2a admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:17.629159 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624e7e9-30e4-4cdd-9357-9cca145cfbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:17.629934 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624e7e9-30e4-4cdd-9357-9cca145cfbf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:17.630226 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624e7e9-30e4-4cdd-9357-9cca145cfbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:17.630545 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624e7e9-30e4-4cdd-9357-9cca145cfbf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:17.631271 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624e7e9-30e4-4cdd-9357-9cca145cfbf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:17.632890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624e7e9-30e4-4cdd-9357-9cca145cfbf2 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:17.643096 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1283/2581] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:14:17.647200 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2b490-cf2c-4b88-b389-21dcf314749b admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:17.657467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2b490-cf2c-4b88-b389-21dcf314749b admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:17.662073 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1b549-660e-442e-b7f5-67c94f91b6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:17.662684 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1b549-660e-442e-b7f5-67c94f91b6cb admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:17.663326 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1b549-660e-442e-b7f5-67c94f91b6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:17.663603 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1b549-660e-442e-b7f5-67c94f91b6cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:17.664872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b1b549-660e-442e-b7f5-67c94f91b6cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:17.666477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b1b549-660e-442e-b7f5-67c94f91b6cb admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:17.677252 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1299/2582] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:14:17.682266 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624e7e9-30e4-4cdd-9357-9cca145cfbf2 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:17.694265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624e7e9-30e4-4cdd-9357-9cca145cfbf2 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:17.699810 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d9ee3-872e-4637-aa38-78521016f2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:17.700574 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d9ee3-872e-4637-aa38-78521016f2a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:17.700922 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d9ee3-872e-4637-aa38-78521016f2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:17.701265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d9ee3-872e-4637-aa38-78521016f2a0 admin admin] PATH_INFO: `/v3/projects/9df5e8e2b1ba460eabcc5d7c95417628/users/e77fcf3056d94965b6d51cba86b8976c/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:17.719089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d9ee3-872e-4637-aa38-78521016f2a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9df5e8e2b1ba460eabcc5d7c95417628, user_id=e77fcf3056d94965b6d51cba86b8976c, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:17.722071 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d9ee3-872e-4637-aa38-78521016f2a0 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:17.736252 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1284/2583] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:14:17 2022] PUT /identity/v3/projects/9df5e8e2b1ba460eabcc5d7c95417628/users/e77fcf3056d94965b6d51cba86b8976c/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:17.749018 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28c650-b6a7-414d-9e60-5e186e3e0e7f None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:17.749318 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28c650-b6a7-414d-9e60-5e186e3e0e7f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:17.749602 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28c650-b6a7-414d-9e60-5e186e3e0e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:17.781431 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-3d28c650-b6a7-414d-9e60-5e186e3e0e7f None None] MFA Rules not processed for user `e77fcf3056d94965b6d51cba86b8976c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:17.782655 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3d28c650-b6a7-414d-9e60-5e186e3e0e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:17.850273 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1300/2584] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:17.855957 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d9ee3-872e-4637-aa38-78521016f2a0 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:17.932496 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d9ee3-872e-4637-aa38-78521016f2a0 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:17.937335 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13cf8b7-67f6-415a-8cff-2f5101c6c021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:17.937999 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cf8b7-67f6-415a-8cff-2f5101c6c021 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:17.938422 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cf8b7-67f6-415a-8cff-2f5101c6c021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:17.938696 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cf8b7-67f6-415a-8cff-2f5101c6c021 admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:17.939396 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13cf8b7-67f6-415a-8cff-2f5101c6c021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:17.941279 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13cf8b7-67f6-415a-8cff-2f5101c6c021 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:17.977955 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1285/2585] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:14:17 2022] POST /identity/v3/projects => generated 367 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:14:17.982146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28c650-b6a7-414d-9e60-5e186e3e0e7f None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:17.994345 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28c650-b6a7-414d-9e60-5e186e3e0e7f None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:18.067335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:18.081670 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172026f-a5ae-4957-889e-e9c1857d8815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:18.082623 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172026f-a5ae-4957-889e-e9c1857d8815 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:18.083267 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172026f-a5ae-4957-889e-e9c1857d8815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:18.083644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172026f-a5ae-4957-889e-e9c1857d8815 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:18.084625 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172026f-a5ae-4957-889e-e9c1857d8815 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:18.087854 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172026f-a5ae-4957-889e-e9c1857d8815 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:18.147431 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13cf8b7-67f6-415a-8cff-2f5101c6c021 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:18.149992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1301/2586] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:14:17 2022] POST /identity/v3/users => generated 290 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 05:14:18.161235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:18.171582 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13cf8b7-67f6-415a-8cff-2f5101c6c021 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:18.184306 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076caf8-914f-4aa5-ae50-a5b1d352b940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:18.184874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076caf8-914f-4aa5-ae50-a5b1d352b940 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:18.190605 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076caf8-914f-4aa5-ae50-a5b1d352b940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:18.190983 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076caf8-914f-4aa5-ae50-a5b1d352b940 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:18.191822 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076caf8-914f-4aa5-ae50-a5b1d352b940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:18.193512 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076caf8-914f-4aa5-ae50-a5b1d352b940 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:18.209595 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1286/2587] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:14:18.214064 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172026f-a5ae-4957-889e-e9c1857d8815 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:18.227496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172026f-a5ae-4957-889e-e9c1857d8815 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:18.237643 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ca071-a468-4f8c-baf5-b09e430d04f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:18.247151 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ca071-a468-4f8c-baf5-b09e430d04f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:18.247561 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ca071-a468-4f8c-baf5-b09e430d04f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:18.249280 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ca071-a468-4f8c-baf5-b09e430d04f5 admin admin] PATH_INFO: `/v3/projects/eb801296ba9449cfb51e0aee37df2017/users/0152281b0d574c21b027a2000f14099f/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:18.293469 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ca071-a468-4f8c-baf5-b09e430d04f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb801296ba9449cfb51e0aee37df2017, user_id=0152281b0d574c21b027a2000f14099f, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:18.296795 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ca071-a468-4f8c-baf5-b09e430d04f5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:18.320743 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1302/2588] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:14:18 2022] PUT /identity/v3/projects/eb801296ba9449cfb51e0aee37df2017/users/0152281b0d574c21b027a2000f14099f/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:18.324943 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076caf8-914f-4aa5-ae50-a5b1d352b940 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:18.336761 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076caf8-914f-4aa5-ae50-a5b1d352b940 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:18.459470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f253d-2285-4d5a-a317-fa4e1386ac04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:18.461854 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f253d-2285-4d5a-a317-fa4e1386ac04 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:18.462150 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f253d-2285-4d5a-a317-fa4e1386ac04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:18.462433 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f253d-2285-4d5a-a317-fa4e1386ac04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:18.463341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f253d-2285-4d5a-a317-fa4e1386ac04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:18.465259 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f253d-2285-4d5a-a317-fa4e1386ac04 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:18.482978 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ca071-a468-4f8c-baf5-b09e430d04f5 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:18.489319 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1287/2589] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:14:18.495824 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ca071-a468-4f8c-baf5-b09e430d04f5 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:18.510820 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f71a96-67d8-49d1-9e3d-0dfd247f7907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:18.511722 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f71a96-67d8-49d1-9e3d-0dfd247f7907 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:18.512166 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f71a96-67d8-49d1-9e3d-0dfd247f7907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:18.512578 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f71a96-67d8-49d1-9e3d-0dfd247f7907 admin admin] PATH_INFO: `/v3/projects/eb801296ba9449cfb51e0aee37df2017/users/0152281b0d574c21b027a2000f14099f/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:18.519265 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f71a96-67d8-49d1-9e3d-0dfd247f7907 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb801296ba9449cfb51e0aee37df2017, user_id=0152281b0d574c21b027a2000f14099f, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:18.522939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f71a96-67d8-49d1-9e3d-0dfd247f7907 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:18.548119 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1303/2590] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:14:18 2022] PUT /identity/v3/projects/eb801296ba9449cfb51e0aee37df2017/users/0152281b0d574c21b027a2000f14099f/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:18.559438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad31a17-3c52-486d-8a7d-f697bb43b8c4 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:18.560186 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad31a17-3c52-486d-8a7d-f697bb43b8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:18.560966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad31a17-3c52-486d-8a7d-f697bb43b8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:18.605064 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-4ad31a17-3c52-486d-8a7d-f697bb43b8c4 None None] MFA Rules not processed for user `0152281b0d574c21b027a2000f14099f`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:18.606835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-4ad31a17-3c52-486d-8a7d-f697bb43b8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:18.706165 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1288/2591] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:18 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:18.719629 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f71a96-67d8-49d1-9e3d-0dfd247f7907 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:18.722560 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-69f71a96-67d8-49d1-9e3d-0dfd247f7907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:18.831168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f71a96-67d8-49d1-9e3d-0dfd247f7907 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:18.838049 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d59baad-85c7-49c7-ba91-b018cd9e5a5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:18.839481 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59baad-85c7-49c7-ba91-b018cd9e5a5a service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:18.841090 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59baad-85c7-49c7-ba91-b018cd9e5a5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:18.841482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59baad-85c7-49c7-ba91-b018cd9e5a5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:18.855912 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d59baad-85c7-49c7-ba91-b018cd9e5a5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:18.857759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d59baad-85c7-49c7-ba91-b018cd9e5a5a service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:18.870761 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1304/2592] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:18 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:18.970899 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad31a17-3c52-486d-8a7d-f697bb43b8c4 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:18.972908 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-4ad31a17-3c52-486d-8a7d-f697bb43b8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:18.995426 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad31a17-3c52-486d-8a7d-f697bb43b8c4 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:19.003314 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a878074-d85f-4bd2-91c2-8dc75c4ec876 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:19.004243 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a878074-d85f-4bd2-91c2-8dc75c4ec876 service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:19.004625 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a878074-d85f-4bd2-91c2-8dc75c4ec876 service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:19.004996 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a878074-d85f-4bd2-91c2-8dc75c4ec876 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:19.127271 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a878074-d85f-4bd2-91c2-8dc75c4ec876 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:19.129084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a878074-d85f-4bd2-91c2-8dc75c4ec876 service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:19.140549 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1289/2593] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:20.733407 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6024316-b70c-4a5e-8ab0-7403928b9a42 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:20.735283 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6024316-b70c-4a5e-8ab0-7403928b9a42 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:20.738600 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6024316-b70c-4a5e-8ab0-7403928b9a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:20.774367 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-f6024316-b70c-4a5e-8ab0-7403928b9a42 None None] MFA Rules not processed for user `30c0ff27c5a84ebd8615fb4608fc3882`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:20.775838 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f6024316-b70c-4a5e-8ab0-7403928b9a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:20.855899 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1305/2594] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:20.859972 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a878074-d85f-4bd2-91c2-8dc75c4ec876 service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:20.861484 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-9a878074-d85f-4bd2-91c2-8dc75c4ec876 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:20.877195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:20.950675 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a878074-d85f-4bd2-91c2-8dc75c4ec876 service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:20.956123 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b9c30-8468-4d8e-bf10-0d2206022874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:20.956385 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b9c30-8468-4d8e-bf10-0d2206022874 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:20.956672 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b9c30-8468-4d8e-bf10-0d2206022874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:20.956947 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b9c30-8468-4d8e-bf10-0d2206022874 admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:20.957660 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b9c30-8468-4d8e-bf10-0d2206022874 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:20.959846 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b9c30-8468-4d8e-bf10-0d2206022874 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:21.005595 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1290/2595] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:14:20 2022] POST /identity/v3/projects => generated 377 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 05:14:21.010334 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6024316-b70c-4a5e-8ab0-7403928b9a42 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:21.026102 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6024316-b70c-4a5e-8ab0-7403928b9a42 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:21.071245 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:21.096290 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ae1cf-89c2-4b8c-9546-6d3177586143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:21.096908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ae1cf-89c2-4b8c-9546-6d3177586143 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:21.097225 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ae1cf-89c2-4b8c-9546-6d3177586143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:21.097498 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ae1cf-89c2-4b8c-9546-6d3177586143 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:21.098197 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ae1cf-89c2-4b8c-9546-6d3177586143 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:21.100093 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ae1cf-89c2-4b8c-9546-6d3177586143 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:21.121111 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1306/2596] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:14:21 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:14:21.125136 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b9c30-8468-4d8e-bf10-0d2206022874 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:21.135704 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b9c30-8468-4d8e-bf10-0d2206022874 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:21.140470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a390e0-7148-44f3-ba7c-af30d9706463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:21.141085 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a390e0-7148-44f3-ba7c-af30d9706463 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:21.141360 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a390e0-7148-44f3-ba7c-af30d9706463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:21.141664 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a390e0-7148-44f3-ba7c-af30d9706463 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:21.142359 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a390e0-7148-44f3-ba7c-af30d9706463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:21.143931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a390e0-7148-44f3-ba7c-af30d9706463 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:21.154590 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1291/2597] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:14:21.158851 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ae1cf-89c2-4b8c-9546-6d3177586143 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:21.170319 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ae1cf-89c2-4b8c-9546-6d3177586143 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:21.174942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af47e31-30c7-4ddd-80f3-c37fef4fffd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:21.175609 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af47e31-30c7-4ddd-80f3-c37fef4fffd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:21.175893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af47e31-30c7-4ddd-80f3-c37fef4fffd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:21.176167 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af47e31-30c7-4ddd-80f3-c37fef4fffd6 admin admin] PATH_INFO: `/v3/projects/4779f6b558784418a131a049703404a6/users/26ad3570fcf342b58eb6d4f6c3177591/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:21.194324 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af47e31-30c7-4ddd-80f3-c37fef4fffd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4779f6b558784418a131a049703404a6, user_id=26ad3570fcf342b58eb6d4f6c3177591, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:21.198095 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af47e31-30c7-4ddd-80f3-c37fef4fffd6 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:21.214392 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1307/2598] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:14:21 2022] PUT /identity/v3/projects/4779f6b558784418a131a049703404a6/users/26ad3570fcf342b58eb6d4f6c3177591/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 05:14:21.218467 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a390e0-7148-44f3-ba7c-af30d9706463 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:21.228977 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a390e0-7148-44f3-ba7c-af30d9706463 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:21.295755 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506671a5-ff92-484c-9b74-bb352e8c2b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:21.296522 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506671a5-ff92-484c-9b74-bb352e8c2b81 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:21.296802 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506671a5-ff92-484c-9b74-bb352e8c2b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:21.297070 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506671a5-ff92-484c-9b74-bb352e8c2b81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:21.297816 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506671a5-ff92-484c-9b74-bb352e8c2b81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:21.299603 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506671a5-ff92-484c-9b74-bb352e8c2b81 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:21.316185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1292/2599] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 05:14:21.319395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af47e31-30c7-4ddd-80f3-c37fef4fffd6 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:21.332679 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af47e31-30c7-4ddd-80f3-c37fef4fffd6 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:21.339484 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760d269-7e16-46e8-b908-cbbb25681e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:21.340109 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760d269-7e16-46e8-b908-cbbb25681e66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:21.340387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760d269-7e16-46e8-b908-cbbb25681e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:21.340660 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760d269-7e16-46e8-b908-cbbb25681e66 admin admin] PATH_INFO: `/v3/projects/4779f6b558784418a131a049703404a6/users/26ad3570fcf342b58eb6d4f6c3177591/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:21.343093 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760d269-7e16-46e8-b908-cbbb25681e66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4779f6b558784418a131a049703404a6, user_id=26ad3570fcf342b58eb6d4f6c3177591, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:21.345927 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760d269-7e16-46e8-b908-cbbb25681e66 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:21.365086 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1308/2600] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:14:21 2022] PUT /identity/v3/projects/4779f6b558784418a131a049703404a6/users/26ad3570fcf342b58eb6d4f6c3177591/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:22.984690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506671a5-ff92-484c-9b74-bb352e8c2b81 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:22.986526 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-506671a5-ff92-484c-9b74-bb352e8c2b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:23.068011 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506671a5-ff92-484c-9b74-bb352e8c2b81 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:23.118787 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:23.155911 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a397b64-a8d5-4e2c-a717-10c0bcd877a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:23.157340 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a397b64-a8d5-4e2c-a717-10c0bcd877a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:23.157764 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a397b64-a8d5-4e2c-a717-10c0bcd877a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:23.158145 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a397b64-a8d5-4e2c-a717-10c0bcd877a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:23.160874 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-4a397b64-a8d5-4e2c-a717-10c0bcd877a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:23.218717 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760d269-7e16-46e8-b908-cbbb25681e66 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:23.291813 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a397b64-a8d5-4e2c-a717-10c0bcd877a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:23.293868 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a397b64-a8d5-4e2c-a717-10c0bcd877a0 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:23.311164 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760d269-7e16-46e8-b908-cbbb25681e66 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:23.313767 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1293/2601] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:23.317301 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08782928-a927-4104-b74f-83d48fb8c6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:23.317907 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08782928-a927-4104-b74f-83d48fb8c6ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:23.318186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08782928-a927-4104-b74f-83d48fb8c6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:23.318801 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08782928-a927-4104-b74f-83d48fb8c6ab admin admin] PATH_INFO: `/v3/users/e77fcf3056d94965b6d51cba86b8976c` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:23.320421 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08782928-a927-4104-b74f-83d48fb8c6ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=e77fcf3056d94965b6d51cba86b8976c)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:23.324903 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08782928-a927-4104-b74f-83d48fb8c6ab admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:23.407347 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-08782928-a927-4104-b74f-83d48fb8c6ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e77fcf3056d94965b6d51cba86b8976c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:23.418344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-08782928-a927-4104-b74f-83d48fb8c6ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e77fcf3056d94965b6d51cba86b8976c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:23.418593 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-08782928-a927-4104-b74f-83d48fb8c6ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e77fcf3056d94965b6d51cba86b8976c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:23.442963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-08782928-a927-4104-b74f-83d48fb8c6ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e77fcf3056d94965b6d51cba86b8976c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:23.443809 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-08782928-a927-4104-b74f-83d48fb8c6ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e77fcf3056d94965b6d51cba86b8976c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:23.468539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1309/2602] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:14:23 2022] DELETE /identity/v3/users/e77fcf3056d94965b6d51cba86b8976c => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:23.470929 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f00986-afc0-4c8c-be49-c2ec7c29b1b8 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:23.471224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f00986-afc0-4c8c-be49-c2ec7c29b1b8 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:23.471500 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f00986-afc0-4c8c-be49-c2ec7c29b1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:23.523138 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-47f00986-afc0-4c8c-be49-c2ec7c29b1b8 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:23.526159 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-47f00986-afc0-4c8c-be49-c2ec7c29b1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:23.652638 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1294/2603] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:23.706177 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08782928-a927-4104-b74f-83d48fb8c6ab admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:23.707651 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-08782928-a927-4104-b74f-83d48fb8c6ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:23.721646 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:23.811950 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08782928-a927-4104-b74f-83d48fb8c6ab admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:23.817749 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141a150-fc48-43a6-8147-e45bb8cfd6a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:23.818409 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141a150-fc48-43a6-8147-e45bb8cfd6a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:23.818689 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141a150-fc48-43a6-8147-e45bb8cfd6a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:23.818966 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141a150-fc48-43a6-8147-e45bb8cfd6a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:23.831885 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c141a150-fc48-43a6-8147-e45bb8cfd6a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:23.833350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c141a150-fc48-43a6-8147-e45bb8cfd6a7 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:23.855397 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1310/2604] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:24.901965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f00986-afc0-4c8c-be49-c2ec7c29b1b8 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:24.904344 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-47f00986-afc0-4c8c-be49-c2ec7c29b1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:24.951731 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f00986-afc0-4c8c-be49-c2ec7c29b1b8 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:24.978417 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3bd8b5-d47e-480d-b4ce-a5ac48d65ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:24.979093 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bd8b5-d47e-480d-b4ce-a5ac48d65ede admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:24.979370 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bd8b5-d47e-480d-b4ce-a5ac48d65ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:24.979642 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bd8b5-d47e-480d-b4ce-a5ac48d65ede admin admin] PATH_INFO: `/v3/projects/9df5e8e2b1ba460eabcc5d7c95417628` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:24.980869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3bd8b5-d47e-480d-b4ce-a5ac48d65ede admin admin] RBAC: Authorizing `identity:delete_project(project_id=9df5e8e2b1ba460eabcc5d7c95417628)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:24.982831 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3bd8b5-d47e-480d-b4ce-a5ac48d65ede admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:25.159055 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-2f3bd8b5-d47e-480d-b4ce-a5ac48d65ede admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9df5e8e2b1ba460eabcc5d7c95417628', 'request_id': 'req-2f3bd8b5-d47e-480d-b4ce-a5ac48d65ede'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:25.176473 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-2f3bd8b5-d47e-480d-b4ce-a5ac48d65ede admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9df5e8e2b1ba460eabcc5d7c95417628', 'request_id': 'req-2f3bd8b5-d47e-480d-b4ce-a5ac48d65ede'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:25.190996 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-2f3bd8b5-d47e-480d-b4ce-a5ac48d65ede admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9df5e8e2b1ba460eabcc5d7c95417628', 'request_id': 'req-2f3bd8b5-d47e-480d-b4ce-a5ac48d65ede'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:25.206623 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:25.218706 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-2f3bd8b5-d47e-480d-b4ce-a5ac48d65ede admin admin] The token cache is being invalidate because project 9df5e8e2b1ba460eabcc5d7c95417628 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:14:25.219121 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-2f3bd8b5-d47e-480d-b4ce-a5ac48d65ede admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:25.219924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-2f3bd8b5-d47e-480d-b4ce-a5ac48d65ede admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:25.228427 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1295/2605] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:14:24 2022] DELETE /identity/v3/projects/9df5e8e2b1ba460eabcc5d7c95417628 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:25.230213 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141a150-fc48-43a6-8147-e45bb8cfd6a7 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:25.243698 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141a150-fc48-43a6-8147-e45bb8cfd6a7 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:25.394983 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b53df-5219-4a32-ad1f-4fb2bfbbb2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:25.419882 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b53df-5219-4a32-ad1f-4fb2bfbbb2ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:25.420891 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b53df-5219-4a32-ad1f-4fb2bfbbb2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:25.421376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b53df-5219-4a32-ad1f-4fb2bfbbb2ce admin admin] PATH_INFO: `/v3/users/0152281b0d574c21b027a2000f14099f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:25.423625 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b53df-5219-4a32-ad1f-4fb2bfbbb2ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=0152281b0d574c21b027a2000f14099f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:25.426864 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b53df-5219-4a32-ad1f-4fb2bfbbb2ce admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:25.542380 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fb1b53df-5219-4a32-ad1f-4fb2bfbbb2ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0152281b0d574c21b027a2000f14099f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:25.564657 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fb1b53df-5219-4a32-ad1f-4fb2bfbbb2ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0152281b0d574c21b027a2000f14099f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:25.565436 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fb1b53df-5219-4a32-ad1f-4fb2bfbbb2ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0152281b0d574c21b027a2000f14099f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:25.611429 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fb1b53df-5219-4a32-ad1f-4fb2bfbbb2ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0152281b0d574c21b027a2000f14099f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:25.613271 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fb1b53df-5219-4a32-ad1f-4fb2bfbbb2ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0152281b0d574c21b027a2000f14099f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:25.644827 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1311/2606] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:14:25 2022] DELETE /identity/v3/users/0152281b0d574c21b027a2000f14099f => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:26.539270 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3bd8b5-d47e-480d-b4ce-a5ac48d65ede admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:26.542348 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-2f3bd8b5-d47e-480d-b4ce-a5ac48d65ede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:26.580695 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3bd8b5-d47e-480d-b4ce-a5ac48d65ede admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:26.689137 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9bc4d8-86bb-4ac4-9bf7-65d3b748332b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:26.690123 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9bc4d8-86bb-4ac4-9bf7-65d3b748332b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:26.690571 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9bc4d8-86bb-4ac4-9bf7-65d3b748332b admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:26.691158 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9bc4d8-86bb-4ac4-9bf7-65d3b748332b admin admin] PATH_INFO: `/v3/projects/eb801296ba9449cfb51e0aee37df2017` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:26.692608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9bc4d8-86bb-4ac4-9bf7-65d3b748332b admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb801296ba9449cfb51e0aee37df2017)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:26.695655 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9bc4d8-86bb-4ac4-9bf7-65d3b748332b admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:26.768949 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9e9bc4d8-86bb-4ac4-9bf7-65d3b748332b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb801296ba9449cfb51e0aee37df2017', 'request_id': 'req-9e9bc4d8-86bb-4ac4-9bf7-65d3b748332b'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:26.796267 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9e9bc4d8-86bb-4ac4-9bf7-65d3b748332b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb801296ba9449cfb51e0aee37df2017', 'request_id': 'req-9e9bc4d8-86bb-4ac4-9bf7-65d3b748332b'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:26.818499 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9e9bc4d8-86bb-4ac4-9bf7-65d3b748332b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb801296ba9449cfb51e0aee37df2017', 'request_id': 'req-9e9bc4d8-86bb-4ac4-9bf7-65d3b748332b'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:26.848787 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9e9bc4d8-86bb-4ac4-9bf7-65d3b748332b admin admin] The token cache is being invalidate because project eb801296ba9449cfb51e0aee37df2017 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:14:26.849174 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9e9bc4d8-86bb-4ac4-9bf7-65d3b748332b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:26.852912 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-9e9bc4d8-86bb-4ac4-9bf7-65d3b748332b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:26.857588 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1296/2607] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:14:26 2022] DELETE /identity/v3/projects/eb801296ba9449cfb51e0aee37df2017 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:26.859605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb167a70-291e-4fe0-8e5e-ccc9684c9b8d None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:26.860079 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb167a70-291e-4fe0-8e5e-ccc9684c9b8d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:26.860694 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb167a70-291e-4fe0-8e5e-ccc9684c9b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:26.946389 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:26.967488 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-eb167a70-291e-4fe0-8e5e-ccc9684c9b8d None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:26.969040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-eb167a70-291e-4fe0-8e5e-ccc9684c9b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:27.070770 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1312/2608] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:27.079149 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90af0b7-73de-45f9-862c-bf314ec7612e None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:27.079711 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90af0b7-73de-45f9-862c-bf314ec7612e None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:27.080174 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90af0b7-73de-45f9-862c-bf314ec7612e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:27.121350 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-f90af0b7-73de-45f9-862c-bf314ec7612e None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:27.122661 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-f90af0b7-73de-45f9-862c-bf314ec7612e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:27.140890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1297/2609] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:27.141799 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb167a70-291e-4fe0-8e5e-ccc9684c9b8d None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:27.160777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb167a70-291e-4fe0-8e5e-ccc9684c9b8d None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:27.174533 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ff00a-676d-423c-a036-faa75b4de39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:27.175527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ff00a-676d-423c-a036-faa75b4de39b admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:27.176222 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ff00a-676d-423c-a036-faa75b4de39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:27.178961 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ff00a-676d-423c-a036-faa75b4de39b admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:27.180754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ff00a-676d-423c-a036-faa75b4de39b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 05:14:27.181338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ff00a-676d-423c-a036-faa75b4de39b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:27.183303 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ff00a-676d-423c-a036-faa75b4de39b admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:27.203850 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1313/2610] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 05:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 05:14:27.208763 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90af0b7-73de-45f9-862c-bf314ec7612e None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:27.230683 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90af0b7-73de-45f9-862c-bf314ec7612e None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:27.237681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53307b21-de06-48e4-9373-64422797a30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:27.238865 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53307b21-de06-48e4-9373-64422797a30c admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:27.239380 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53307b21-de06-48e4-9373-64422797a30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:27.239867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53307b21-de06-48e4-9373-64422797a30c admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:27.240637 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53307b21-de06-48e4-9373-64422797a30c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:27.244061 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53307b21-de06-48e4-9373-64422797a30c admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:27.303637 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1298/2611] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:14:27 2022] POST /identity/v3/projects => generated 387 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:14:27.305869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ff00a-676d-423c-a036-faa75b4de39b admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:27.322256 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ff00a-676d-423c-a036-faa75b4de39b admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:27.433961 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab0460-58be-4db6-a09f-5050565ce791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:27.434922 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab0460-58be-4db6-a09f-5050565ce791 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:27.435578 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab0460-58be-4db6-a09f-5050565ce791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:27.436067 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab0460-58be-4db6-a09f-5050565ce791 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:27.436962 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab0460-58be-4db6-a09f-5050565ce791 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:27.440192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab0460-58be-4db6-a09f-5050565ce791 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:27.509703 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1314/2612] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:14:27 2022] POST /identity/v3/users => generated 294 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:14:27.513720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53307b21-de06-48e4-9373-64422797a30c admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:27.537202 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53307b21-de06-48e4-9373-64422797a30c admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:27.550315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a34b6e-4c6f-4f02-b42b-533c628d042d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:27.551088 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a34b6e-4c6f-4f02-b42b-533c628d042d admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:27.551227 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a34b6e-4c6f-4f02-b42b-533c628d042d admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:27.559730 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a34b6e-4c6f-4f02-b42b-533c628d042d admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:27.560788 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a34b6e-4c6f-4f02-b42b-533c628d042d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:27.563042 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a34b6e-4c6f-4f02-b42b-533c628d042d admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:27.589324 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1299/2613] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:14:27.593633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab0460-58be-4db6-a09f-5050565ce791 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:27.611175 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab0460-58be-4db6-a09f-5050565ce791 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:27.622623 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a77976-8a51-4419-bdfc-c7674d430cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:27.623361 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a77976-8a51-4419-bdfc-c7674d430cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:27.623709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a77976-8a51-4419-bdfc-c7674d430cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:27.624058 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a77976-8a51-4419-bdfc-c7674d430cc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:27.624841 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a77976-8a51-4419-bdfc-c7674d430cc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:27.626594 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a77976-8a51-4419-bdfc-c7674d430cc2 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:27.655342 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1315/2614] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:14:27.656531 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a34b6e-4c6f-4f02-b42b-533c628d042d admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:27.672000 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a34b6e-4c6f-4f02-b42b-533c628d042d admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:27.678607 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbec3a-4268-410b-8ade-fec02f28494d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:27.679356 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbec3a-4268-410b-8ade-fec02f28494d admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:27.679672 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbec3a-4268-410b-8ade-fec02f28494d admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:27.680049 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbec3a-4268-410b-8ade-fec02f28494d admin admin] PATH_INFO: `/v3/projects/89d27795d45c423282b441c803564607/users/58619a152efa4aea831b88cbd43ad356/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:27.707713 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bbec3a-4268-410b-8ade-fec02f28494d admin admin] RBAC: Authorizing `identity:create_grant(project_id=89d27795d45c423282b441c803564607, user_id=58619a152efa4aea831b88cbd43ad356, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:27.711149 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bbec3a-4268-410b-8ade-fec02f28494d admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:27.738088 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1300/2615] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:14:27 2022] PUT /identity/v3/projects/89d27795d45c423282b441c803564607/users/58619a152efa4aea831b88cbd43ad356/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:27.751676 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbc895-8db0-459a-a997-6c12770e7bf1 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:27.752010 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbc895-8db0-459a-a997-6c12770e7bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:27.752301 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbc895-8db0-459a-a997-6c12770e7bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:27.818600 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-a7dbc895-8db0-459a-a997-6c12770e7bf1 None None] MFA Rules not processed for user `58619a152efa4aea831b88cbd43ad356`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:27.821576 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a7dbc895-8db0-459a-a997-6c12770e7bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:28.001517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1316/2616] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:28.005899 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbec3a-4268-410b-8ade-fec02f28494d admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:28.088163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:28.166684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbec3a-4268-410b-8ade-fec02f28494d admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:28.173377 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639925be-6cc9-46b4-bf47-c3a3d76b6422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:28.174569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639925be-6cc9-46b4-bf47-c3a3d76b6422 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:28.175112 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639925be-6cc9-46b4-bf47-c3a3d76b6422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:28.175452 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639925be-6cc9-46b4-bf47-c3a3d76b6422 admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:28.176262 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639925be-6cc9-46b4-bf47-c3a3d76b6422 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:28.178489 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639925be-6cc9-46b4-bf47-c3a3d76b6422 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:28.198414 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a63d7f-f930-47c5-abd3-21a562a950c5 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:28.198743 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a63d7f-f930-47c5-abd3-21a562a950c5 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:28.199399 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a63d7f-f930-47c5-abd3-21a562a950c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:28.208877 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:28.213935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1301/2617] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:14:28 2022] POST /identity/v3/projects => generated 387 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:14:28.217255 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639925be-6cc9-46b4-bf47-c3a3d76b6422 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:28.244777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-80a63d7f-f930-47c5-abd3-21a562a950c5 None None] MFA Rules not processed for user `26ad3570fcf342b58eb6d4f6c3177591`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:28.246033 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-80a63d7f-f930-47c5-abd3-21a562a950c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:28.300610 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639925be-6cc9-46b4-bf47-c3a3d76b6422 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:28.310543 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54566dbf-9e62-4e84-85a8-01be61cf5d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:28.311539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54566dbf-9e62-4e84-85a8-01be61cf5d4a admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:28.312182 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54566dbf-9e62-4e84-85a8-01be61cf5d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:28.312562 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54566dbf-9e62-4e84-85a8-01be61cf5d4a admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:28.315260 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54566dbf-9e62-4e84-85a8-01be61cf5d4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:28.317472 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54566dbf-9e62-4e84-85a8-01be61cf5d4a admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:28.361333 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1302/2618] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:14:28 2022] POST /identity/v3/users => generated 300 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:14:28.365379 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54566dbf-9e62-4e84-85a8-01be61cf5d4a admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:28.378668 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54566dbf-9e62-4e84-85a8-01be61cf5d4a admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:28.389289 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73822b7-f85b-463f-8389-0e8c0842ea4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:28.390382 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73822b7-f85b-463f-8389-0e8c0842ea4e admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:28.390931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73822b7-f85b-463f-8389-0e8c0842ea4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:28.391368 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73822b7-f85b-463f-8389-0e8c0842ea4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:28.392212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73822b7-f85b-463f-8389-0e8c0842ea4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:28.394146 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73822b7-f85b-463f-8389-0e8c0842ea4e admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:28.395607 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1317/2619] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 05:14:28.414508 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a63d7f-f930-47c5-abd3-21a562a950c5 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:28.420851 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-80a63d7f-f930-47c5-abd3-21a562a950c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:28.426442 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1303/2620] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 05:14:28.430763 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73822b7-f85b-463f-8389-0e8c0842ea4e admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:28.455900 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73822b7-f85b-463f-8389-0e8c0842ea4e admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:28.469430 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766259c7-9837-42ef-a87c-72cb2bda2a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:28.470121 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766259c7-9837-42ef-a87c-72cb2bda2a0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:28.470462 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766259c7-9837-42ef-a87c-72cb2bda2a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:28.470659 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766259c7-9837-42ef-a87c-72cb2bda2a0e admin admin] PATH_INFO: `/v3/projects/7201c78dbe494525a697c1439d8fc351/users/e158d1b7cd8544c881f37a7a738f272f/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:28.498617 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766259c7-9837-42ef-a87c-72cb2bda2a0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7201c78dbe494525a697c1439d8fc351, user_id=e158d1b7cd8544c881f37a7a738f272f, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:28.504515 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766259c7-9837-42ef-a87c-72cb2bda2a0e admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:28.536914 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1304/2621] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:14:28 2022] PUT /identity/v3/projects/7201c78dbe494525a697c1439d8fc351/users/e158d1b7cd8544c881f37a7a738f272f/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:28.542393 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766259c7-9837-42ef-a87c-72cb2bda2a0e admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:28.561375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a63d7f-f930-47c5-abd3-21a562a950c5 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:28.673791 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce71c0a-bdb2-4a1c-af88-45ca52c0bf5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:28.674529 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce71c0a-bdb2-4a1c-af88-45ca52c0bf5e service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:28.675193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce71c0a-bdb2-4a1c-af88-45ca52c0bf5e service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:28.675545 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce71c0a-bdb2-4a1c-af88-45ca52c0bf5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:28.678723 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-5ce71c0a-bdb2-4a1c-af88-45ca52c0bf5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:28.683440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766259c7-9837-42ef-a87c-72cb2bda2a0e admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:28.693915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8896f4a2-7162-4ed2-a05b-a3ca0a64250b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:28.694912 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896f4a2-7162-4ed2-a05b-a3ca0a64250b admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:28.696871 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896f4a2-7162-4ed2-a05b-a3ca0a64250b admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:28.697461 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896f4a2-7162-4ed2-a05b-a3ca0a64250b admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:28.698475 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8896f4a2-7162-4ed2-a05b-a3ca0a64250b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:28.702460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8896f4a2-7162-4ed2-a05b-a3ca0a64250b admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:28.711956 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:28.726400 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1305/2622] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 05:14:28.727956 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8896f4a2-7162-4ed2-a05b-a3ca0a64250b admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:28.737145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:28.743109 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8896f4a2-7162-4ed2-a05b-a3ca0a64250b admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:28.752140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0340d7-d313-4d0c-abd1-11e8bd4948d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:28.753184 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0340d7-d313-4d0c-abd1-11e8bd4948d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:28.753835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0340d7-d313-4d0c-abd1-11e8bd4948d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:28.754505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0340d7-d313-4d0c-abd1-11e8bd4948d2 admin admin] PATH_INFO: `/v3/projects/7201c78dbe494525a697c1439d8fc351/users/e158d1b7cd8544c881f37a7a738f272f/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:28.756975 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0340d7-d313-4d0c-abd1-11e8bd4948d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7201c78dbe494525a697c1439d8fc351, user_id=e158d1b7cd8544c881f37a7a738f272f, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:28.760186 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0340d7-d313-4d0c-abd1-11e8bd4948d2 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:28.772711 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1306/2623] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:14:28 2022] PUT /identity/v3/projects/7201c78dbe494525a697c1439d8fc351/users/e158d1b7cd8544c881f37a7a738f272f/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:28.787157 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4cd35-830e-4ff1-97e3-082aad2a2bfa None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:28.787447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4cd35-830e-4ff1-97e3-082aad2a2bfa None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:28.787724 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4cd35-830e-4ff1-97e3-082aad2a2bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:28.799202 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce71c0a-bdb2-4a1c-af88-45ca52c0bf5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:28.801835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce71c0a-bdb2-4a1c-af88-45ca52c0bf5e service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:28.817626 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1318/2624] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:28.856589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-c4d4cd35-830e-4ff1-97e3-082aad2a2bfa None None] MFA Rules not processed for user `e158d1b7cd8544c881f37a7a738f272f`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:28.858373 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c4d4cd35-830e-4ff1-97e3-082aad2a2bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:29.037162 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1307/2625] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:28 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:29.050606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce71c0a-bdb2-4a1c-af88-45ca52c0bf5e service nova] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:29.052609 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-5ce71c0a-bdb2-4a1c-af88-45ca52c0bf5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:29.113653 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4cd35-830e-4ff1-97e3-082aad2a2bfa None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:29.115446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c4d4cd35-830e-4ff1-97e3-082aad2a2bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:29.206068 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce71c0a-bdb2-4a1c-af88-45ca52c0bf5e service nova] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:29.275207 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4cd35-830e-4ff1-97e3-082aad2a2bfa None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:29.282525 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7939f865-5af3-43ca-92ba-6444ad658e01 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350f72f740f749f68175c1c4e8ae9d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:29.283132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde954e-c9fc-49bf-8d18-6740adaedc13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:29.283320 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939f865-5af3-43ca-92ba-6444ad658e01 service placement] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:29.283672 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939f865-5af3-43ca-92ba-6444ad658e01 service placement] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:29.283963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde954e-c9fc-49bf-8d18-6740adaedc13 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:29.284086 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939f865-5af3-43ca-92ba-6444ad658e01 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:29.284365 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde954e-c9fc-49bf-8d18-6740adaedc13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:29.284706 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde954e-c9fc-49bf-8d18-6740adaedc13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:29.286635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-7939f865-5af3-43ca-92ba-6444ad658e01 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:29.297719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde954e-c9fc-49bf-8d18-6740adaedc13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:29.300197 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde954e-c9fc-49bf-8d18-6740adaedc13 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:29.302475 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7939f865-5af3-43ca-92ba-6444ad658e01 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:29.304763 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7939f865-5af3-43ca-92ba-6444ad658e01 service placement] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:29.315254 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1319/2626] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:29 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:29.316957 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde954e-c9fc-49bf-8d18-6740adaedc13 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:29.318469 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7dde954e-c9fc-49bf-8d18-6740adaedc13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:29.320988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1308/2627] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:29.359798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7939f865-5af3-43ca-92ba-6444ad658e01 service placement] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:29.475022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde954e-c9fc-49bf-8d18-6740adaedc13 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:29.481973 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc773b-2ec4-4fe0-9d84-be761cd1ce96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:29.483419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc773b-2ec4-4fe0-9d84-be761cd1ce96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:29.483551 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc773b-2ec4-4fe0-9d84-be761cd1ce96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:29.483802 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc773b-2ec4-4fe0-9d84-be761cd1ce96 admin admin] PATH_INFO: `/v3/users/40a2cbbc654749199e742ae96329a1d9` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:29.485157 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc773b-2ec4-4fe0-9d84-be761cd1ce96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40a2cbbc654749199e742ae96329a1d9)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:29.488010 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc773b-2ec4-4fe0-9d84-be761cd1ce96 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:29.514657 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7939f865-5af3-43ca-92ba-6444ad658e01 service placement] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:29.520536 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134ef30-baee-4266-8b7e-a8c6561b1de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:29.521928 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134ef30-baee-4266-8b7e-a8c6561b1de4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:29.521928 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134ef30-baee-4266-8b7e-a8c6561b1de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:29.522168 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134ef30-baee-4266-8b7e-a8c6561b1de4 admin admin] PATH_INFO: `/v3/users/58619a152efa4aea831b88cbd43ad356` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:29.523497 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2134ef30-baee-4266-8b7e-a8c6561b1de4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58619a152efa4aea831b88cbd43ad356)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:29.525686 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2134ef30-baee-4266-8b7e-a8c6561b1de4 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:29.528648 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:29.541779 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:29.609190 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-9abc773b-2ec4-4fe0-9d84-be761cd1ce96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40a2cbbc654749199e742ae96329a1d9'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:29.628857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-9abc773b-2ec4-4fe0-9d84-be761cd1ce96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40a2cbbc654749199e742ae96329a1d9'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:29.632307 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-9abc773b-2ec4-4fe0-9d84-be761cd1ce96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40a2cbbc654749199e742ae96329a1d9'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:29.636152 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-2134ef30-baee-4266-8b7e-a8c6561b1de4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58619a152efa4aea831b88cbd43ad356'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:29.637584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-2134ef30-baee-4266-8b7e-a8c6561b1de4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58619a152efa4aea831b88cbd43ad356'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:29.659137 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-2134ef30-baee-4266-8b7e-a8c6561b1de4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58619a152efa4aea831b88cbd43ad356'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:29.673323 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-2134ef30-baee-4266-8b7e-a8c6561b1de4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58619a152efa4aea831b88cbd43ad356'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:29.674853 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-2134ef30-baee-4266-8b7e-a8c6561b1de4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58619a152efa4aea831b88cbd43ad356'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:29.676499 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-9abc773b-2ec4-4fe0-9d84-be761cd1ce96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40a2cbbc654749199e742ae96329a1d9'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:29.679546 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-9abc773b-2ec4-4fe0-9d84-be761cd1ce96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40a2cbbc654749199e742ae96329a1d9'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:29.696406 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1309/2628] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:14:29 2022] DELETE /identity/v3/users/58619a152efa4aea831b88cbd43ad356 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 05:14:29.705947 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1320/2629] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:14:29 2022] DELETE /identity/v3/users/40a2cbbc654749199e742ae96329a1d9 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:29.706644 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827171c2-6219-4283-a145-9be925d883a2 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:29.707013 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827171c2-6219-4283-a145-9be925d883a2 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:29.707557 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827171c2-6219-4283-a145-9be925d883a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:29.733355 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-827171c2-6219-4283-a145-9be925d883a2 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:29.734541 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-827171c2-6219-4283-a145-9be925d883a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:29.817607 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1310/2630] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:29.829358 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc773b-2ec4-4fe0-9d84-be761cd1ce96 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:29.831608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9abc773b-2ec4-4fe0-9d84-be761cd1ce96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:29.912568 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc773b-2ec4-4fe0-9d84-be761cd1ce96 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:29.919149 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebfe987-cc56-495e-ba0a-844347d7a30c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:29.919853 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfe987-cc56-495e-ba0a-844347d7a30c service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:29.920312 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfe987-cc56-495e-ba0a-844347d7a30c service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:29.920621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfe987-cc56-495e-ba0a-844347d7a30c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:29.938500 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebfe987-cc56-495e-ba0a-844347d7a30c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:29.940352 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebfe987-cc56-495e-ba0a-844347d7a30c service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:29.952748 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1321/2631] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:29.962315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827171c2-6219-4283-a145-9be925d883a2 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:29.964084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-827171c2-6219-4283-a145-9be925d883a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:29.981365 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827171c2-6219-4283-a145-9be925d883a2 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:29.989929 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fac181-efa0-442a-b2f0-ecd3fbb60ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:29.990645 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fac181-efa0-442a-b2f0-ecd3fbb60ef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:29.991081 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fac181-efa0-442a-b2f0-ecd3fbb60ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:29.991627 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fac181-efa0-442a-b2f0-ecd3fbb60ef1 admin admin] PATH_INFO: `/v3/projects/e43171bfd4564b0c92395767641b91ce` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:29.993502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fac181-efa0-442a-b2f0-ecd3fbb60ef1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e43171bfd4564b0c92395767641b91ce)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:29.996276 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fac181-efa0-442a-b2f0-ecd3fbb60ef1 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:30.062862 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-26fac181-efa0-442a-b2f0-ecd3fbb60ef1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e43171bfd4564b0c92395767641b91ce', 'request_id': 'req-26fac181-efa0-442a-b2f0-ecd3fbb60ef1'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:30.073632 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-26fac181-efa0-442a-b2f0-ecd3fbb60ef1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e43171bfd4564b0c92395767641b91ce', 'request_id': 'req-26fac181-efa0-442a-b2f0-ecd3fbb60ef1'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:30.084915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-26fac181-efa0-442a-b2f0-ecd3fbb60ef1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e43171bfd4564b0c92395767641b91ce', 'request_id': 'req-26fac181-efa0-442a-b2f0-ecd3fbb60ef1'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:30.108965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-26fac181-efa0-442a-b2f0-ecd3fbb60ef1 admin admin] The token cache is being invalidate because project e43171bfd4564b0c92395767641b91ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:14:30.109259 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-26fac181-efa0-442a-b2f0-ecd3fbb60ef1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:30.112685 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-26fac181-efa0-442a-b2f0-ecd3fbb60ef1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:30.121730 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1311/2632] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:14:29 2022] DELETE /identity/v3/projects/e43171bfd4564b0c92395767641b91ce => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:30.123296 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca557b2f-1ba0-4968-9de9-5bbba5c4083b None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:30.123804 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca557b2f-1ba0-4968-9de9-5bbba5c4083b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:30.124093 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca557b2f-1ba0-4968-9de9-5bbba5c4083b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:30.151461 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ca557b2f-1ba0-4968-9de9-5bbba5c4083b None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:30.153647 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ca557b2f-1ba0-4968-9de9-5bbba5c4083b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:30.248899 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1322/2633] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:30.256685 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842787af-e79e-47d0-9211-7cb183b9bfb8 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:30.257000 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842787af-e79e-47d0-9211-7cb183b9bfb8 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:30.257824 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842787af-e79e-47d0-9211-7cb183b9bfb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:30.270206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:30.283953 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-842787af-e79e-47d0-9211-7cb183b9bfb8 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:30.286545 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-842787af-e79e-47d0-9211-7cb183b9bfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:30.300627 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1312/2634] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:30.305380 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca557b2f-1ba0-4968-9de9-5bbba5c4083b None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:30.319340 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca557b2f-1ba0-4968-9de9-5bbba5c4083b None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:30.329590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ce1d2-55fb-449e-9882-b3976e1ad716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:30.330343 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ce1d2-55fb-449e-9882-b3976e1ad716 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:30.330829 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ce1d2-55fb-449e-9882-b3976e1ad716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:30.330966 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ce1d2-55fb-449e-9882-b3976e1ad716 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:30.331808 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ce1d2-55fb-449e-9882-b3976e1ad716 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 05:14:30.332477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ce1d2-55fb-449e-9882-b3976e1ad716 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:30.334180 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ce1d2-55fb-449e-9882-b3976e1ad716 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:30.369442 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1323/2635] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 05:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 05:14:30.373908 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842787af-e79e-47d0-9211-7cb183b9bfb8 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:30.385381 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842787af-e79e-47d0-9211-7cb183b9bfb8 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:30.406870 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b975c66-5cd1-486a-967f-953d777ed70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:30.407586 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b975c66-5cd1-486a-967f-953d777ed70a admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:30.407783 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b975c66-5cd1-486a-967f-953d777ed70a admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:30.408030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b975c66-5cd1-486a-967f-953d777ed70a admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:30.409684 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b975c66-5cd1-486a-967f-953d777ed70a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:30.412017 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b975c66-5cd1-486a-967f-953d777ed70a admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:30.441897 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ce1d2-55fb-449e-9882-b3976e1ad716 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:30.444218 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-453ce1d2-55fb-449e-9882-b3976e1ad716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:30.469803 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ce1d2-55fb-449e-9882-b3976e1ad716 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:30.483846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb3991-5b0f-4dc1-b06f-e9bf58dd4f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:30.484591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1313/2636] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:14:30 2022] POST /identity/v3/projects => generated 389 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:14:30.487849 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb3991-5b0f-4dc1-b06f-e9bf58dd4f99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:30.488151 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb3991-5b0f-4dc1-b06f-e9bf58dd4f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:30.488489 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb3991-5b0f-4dc1-b06f-e9bf58dd4f99 admin admin] PATH_INFO: `/v3/projects/89d27795d45c423282b441c803564607` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:30.488906 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b975c66-5cd1-486a-967f-953d777ed70a admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:30.489984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb3991-5b0f-4dc1-b06f-e9bf58dd4f99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89d27795d45c423282b441c803564607)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:30.497276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb3991-5b0f-4dc1-b06f-e9bf58dd4f99 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:30.571753 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:30.611211 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d8cb3991-5b0f-4dc1-b06f-e9bf58dd4f99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89d27795d45c423282b441c803564607', 'request_id': 'req-d8cb3991-5b0f-4dc1-b06f-e9bf58dd4f99'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:30.617139 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b975c66-5cd1-486a-967f-953d777ed70a admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:30.625995 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d8cb3991-5b0f-4dc1-b06f-e9bf58dd4f99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89d27795d45c423282b441c803564607', 'request_id': 'req-d8cb3991-5b0f-4dc1-b06f-e9bf58dd4f99'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:30.644487 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d8cb3991-5b0f-4dc1-b06f-e9bf58dd4f99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89d27795d45c423282b441c803564607', 'request_id': 'req-d8cb3991-5b0f-4dc1-b06f-e9bf58dd4f99'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:30.660519 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d8cb3991-5b0f-4dc1-b06f-e9bf58dd4f99 admin admin] The token cache is being invalidate because project 89d27795d45c423282b441c803564607 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:14:30.660982 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d8cb3991-5b0f-4dc1-b06f-e9bf58dd4f99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:30.661928 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d8cb3991-5b0f-4dc1-b06f-e9bf58dd4f99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:30.664259 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1324/2637] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:14:30 2022] DELETE /identity/v3/projects/89d27795d45c423282b441c803564607 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 05:14:30.668718 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb3991-5b0f-4dc1-b06f-e9bf58dd4f99 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:30.670335 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d8cb3991-5b0f-4dc1-b06f-e9bf58dd4f99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:30.783377 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a695c03-f0de-461a-8244-05aa86a4aa0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:30.789627 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb3991-5b0f-4dc1-b06f-e9bf58dd4f99 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:30.793171 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a695c03-f0de-461a-8244-05aa86a4aa0d admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:30.793475 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a695c03-f0de-461a-8244-05aa86a4aa0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:30.793790 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a695c03-f0de-461a-8244-05aa86a4aa0d admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:30.794492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a695c03-f0de-461a-8244-05aa86a4aa0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:30.796162 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ad1e0-928b-4bdd-9894-343a9c505bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:30.801769 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a695c03-f0de-461a-8244-05aa86a4aa0d admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:30.801978 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ad1e0-928b-4bdd-9894-343a9c505bbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:30.802657 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ad1e0-928b-4bdd-9894-343a9c505bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:30.803230 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ad1e0-928b-4bdd-9894-343a9c505bbb admin admin] PATH_INFO: `/v3/users/e158d1b7cd8544c881f37a7a738f272f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:30.806006 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ad1e0-928b-4bdd-9894-343a9c505bbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e158d1b7cd8544c881f37a7a738f272f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:30.809376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ad1e0-928b-4bdd-9894-343a9c505bbb admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:30.872383 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1314/2638] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:14:30 2022] POST /identity/v3/users => generated 295 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:14:30.876146 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a695c03-f0de-461a-8244-05aa86a4aa0d admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:30.880987 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c57ad1e0-928b-4bdd-9894-343a9c505bbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e158d1b7cd8544c881f37a7a738f272f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:30.882584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-6a695c03-f0de-461a-8244-05aa86a4aa0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:30.890780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c57ad1e0-928b-4bdd-9894-343a9c505bbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e158d1b7cd8544c881f37a7a738f272f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:30.891695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c57ad1e0-928b-4bdd-9894-343a9c505bbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e158d1b7cd8544c881f37a7a738f272f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:30.907052 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c57ad1e0-928b-4bdd-9894-343a9c505bbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e158d1b7cd8544c881f37a7a738f272f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:30.907972 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c57ad1e0-928b-4bdd-9894-343a9c505bbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e158d1b7cd8544c881f37a7a738f272f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:30.926435 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a695c03-f0de-461a-8244-05aa86a4aa0d admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:30.929941 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1325/2639] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:14:30 2022] DELETE /identity/v3/users/e158d1b7cd8544c881f37a7a738f272f => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:31.062426 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efac6ca-0be0-4bb0-80c8-2200150c8e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:31.063058 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efac6ca-0be0-4bb0-80c8-2200150c8e7a admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:31.063771 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efac6ca-0be0-4bb0-80c8-2200150c8e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:31.063771 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efac6ca-0be0-4bb0-80c8-2200150c8e7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:31.064318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efac6ca-0be0-4bb0-80c8-2200150c8e7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:31.066224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efac6ca-0be0-4bb0-80c8-2200150c8e7a admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:31.083298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1315/2640] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:14:31.087446 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ad1e0-928b-4bdd-9894-343a9c505bbb admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:31.089331 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c57ad1e0-928b-4bdd-9894-343a9c505bbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:31.104988 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ad1e0-928b-4bdd-9894-343a9c505bbb admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:31.113916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952826cb-430e-4d9c-bbcd-b4c65a2e81f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:31.114663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952826cb-430e-4d9c-bbcd-b4c65a2e81f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:31.114947 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952826cb-430e-4d9c-bbcd-b4c65a2e81f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:31.115380 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952826cb-430e-4d9c-bbcd-b4c65a2e81f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:31.116211 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952826cb-430e-4d9c-bbcd-b4c65a2e81f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:31.119036 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952826cb-430e-4d9c-bbcd-b4c65a2e81f7 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:31.144569 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1326/2641] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 05:14:31.146205 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efac6ca-0be0-4bb0-80c8-2200150c8e7a admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:31.165238 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efac6ca-0be0-4bb0-80c8-2200150c8e7a admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:31.186529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd5fe4-cf97-44c0-a1a2-10cc3bc49886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:31.187343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd5fe4-cf97-44c0-a1a2-10cc3bc49886 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:31.187899 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd5fe4-cf97-44c0-a1a2-10cc3bc49886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:31.188191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd5fe4-cf97-44c0-a1a2-10cc3bc49886 admin admin] PATH_INFO: `/v3/projects/c27540203cac4353859078edecd89b4d/users/28bf7f4d000248c9b40c1b4e596b5d3a/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:31.195590 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:31.214271 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd5fe4-cf97-44c0-a1a2-10cc3bc49886 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c27540203cac4353859078edecd89b4d, user_id=28bf7f4d000248c9b40c1b4e596b5d3a, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:31.218395 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd5fe4-cf97-44c0-a1a2-10cc3bc49886 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:31.239726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1316/2642] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:14:31 2022] PUT /identity/v3/projects/c27540203cac4353859078edecd89b4d/users/28bf7f4d000248c9b40c1b4e596b5d3a/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:31.244385 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8201b-10d8-476b-8b49-0c46ee15b0bc None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:31.244715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8201b-10d8-476b-8b49-0c46ee15b0bc None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:31.244970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8201b-10d8-476b-8b49-0c46ee15b0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:31.276596 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-0ef8201b-10d8-476b-8b49-0c46ee15b0bc None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:31.278025 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0ef8201b-10d8-476b-8b49-0c46ee15b0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:31.318279 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:31.331125 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd5fe4-cf97-44c0-a1a2-10cc3bc49886 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:31.334865 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-0ccd5fe4-cf97-44c0-a1a2-10cc3bc49886 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:31.405826 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1327/2643] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:31.417979 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8201b-10d8-476b-8b49-0c46ee15b0bc None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:31.421428 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0ef8201b-10d8-476b-8b49-0c46ee15b0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:31.494772 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd5fe4-cf97-44c0-a1a2-10cc3bc49886 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:31.500735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d4a59-9099-4a0a-b26e-5e8683d05c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:31.501519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d4a59-9099-4a0a-b26e-5e8683d05c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:31.501938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d4a59-9099-4a0a-b26e-5e8683d05c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:31.502343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d4a59-9099-4a0a-b26e-5e8683d05c2f admin admin] PATH_INFO: `/v3/projects/7201c78dbe494525a697c1439d8fc351` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:31.503658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d4a59-9099-4a0a-b26e-5e8683d05c2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7201c78dbe494525a697c1439d8fc351)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:31.506553 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d4a59-9099-4a0a-b26e-5e8683d05c2f admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:31.520022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8201b-10d8-476b-8b49-0c46ee15b0bc None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:31.525359 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe8a28f-c4fd-45fc-ba3d-51a6d7af399f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:31.525923 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8a28f-c4fd-45fc-ba3d-51a6d7af399f service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:31.526278 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8a28f-c4fd-45fc-ba3d-51a6d7af399f service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:31.526627 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8a28f-c4fd-45fc-ba3d-51a6d7af399f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:31.541910 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe8a28f-c4fd-45fc-ba3d-51a6d7af399f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:31.543545 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe8a28f-c4fd-45fc-ba3d-51a6d7af399f service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:31.555273 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1328/2644] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:31.583280 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-407d4a59-9099-4a0a-b26e-5e8683d05c2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7201c78dbe494525a697c1439d8fc351', 'request_id': 'req-407d4a59-9099-4a0a-b26e-5e8683d05c2f'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:31.594903 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-407d4a59-9099-4a0a-b26e-5e8683d05c2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7201c78dbe494525a697c1439d8fc351', 'request_id': 'req-407d4a59-9099-4a0a-b26e-5e8683d05c2f'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:31.603276 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-407d4a59-9099-4a0a-b26e-5e8683d05c2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7201c78dbe494525a697c1439d8fc351', 'request_id': 'req-407d4a59-9099-4a0a-b26e-5e8683d05c2f'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:31.619097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-407d4a59-9099-4a0a-b26e-5e8683d05c2f admin admin] The token cache is being invalidate because project 7201c78dbe494525a697c1439d8fc351 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:14:31.619438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-407d4a59-9099-4a0a-b26e-5e8683d05c2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:31.620127 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-407d4a59-9099-4a0a-b26e-5e8683d05c2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:31.622479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1317/2645] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:14:31 2022] DELETE /identity/v3/projects/7201c78dbe494525a697c1439d8fc351 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:31.628314 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8dd915-3d52-4099-a90e-520844fcc1e6 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:31.628688 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8dd915-3d52-4099-a90e-520844fcc1e6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:31.629062 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8dd915-3d52-4099-a90e-520844fcc1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:31.659442 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-5c8dd915-3d52-4099-a90e-520844fcc1e6 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:31.662654 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-5c8dd915-3d52-4099-a90e-520844fcc1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:31.747833 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1329/2646] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 05:14:31.758441 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60df63-2753-48e7-bdd2-8b8fbdcbba91 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:31.758768 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60df63-2753-48e7-bdd2-8b8fbdcbba91 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:31.759328 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60df63-2753-48e7-bdd2-8b8fbdcbba91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:31.789529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-5f60df63-2753-48e7-bdd2-8b8fbdcbba91 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:31.791018 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5f60df63-2753-48e7-bdd2-8b8fbdcbba91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:31.805600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1318/2647] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:31.808617 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8dd915-3d52-4099-a90e-520844fcc1e6 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:31.828688 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8dd915-3d52-4099-a90e-520844fcc1e6 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:31.835917 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d13cb5-f59e-41be-b7b2-38336f2ddee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:31.836875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d13cb5-f59e-41be-b7b2-38336f2ddee5 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:31.837525 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d13cb5-f59e-41be-b7b2-38336f2ddee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:31.837971 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d13cb5-f59e-41be-b7b2-38336f2ddee5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:31.838870 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d13cb5-f59e-41be-b7b2-38336f2ddee5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 05:14:31.839392 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d13cb5-f59e-41be-b7b2-38336f2ddee5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:31.841413 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d13cb5-f59e-41be-b7b2-38336f2ddee5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:31.861794 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1330/2648] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 05:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 05:14:31.862987 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60df63-2753-48e7-bdd2-8b8fbdcbba91 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:31.873318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60df63-2753-48e7-bdd2-8b8fbdcbba91 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:31.878453 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10730a-f916-4c5d-8d9c-b1d11f523bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:31.879068 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10730a-f916-4c5d-8d9c-b1d11f523bd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:31.879348 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10730a-f916-4c5d-8d9c-b1d11f523bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:31.879616 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10730a-f916-4c5d-8d9c-b1d11f523bd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:31.880732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc10730a-f916-4c5d-8d9c-b1d11f523bd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:31.883658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc10730a-f916-4c5d-8d9c-b1d11f523bd7 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:31.931069 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1319/2649] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:14:31 2022] POST /identity/v3/projects => generated 381 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 05:14:31.935219 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d13cb5-f59e-41be-b7b2-38336f2ddee5 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:31.989490 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d13cb5-f59e-41be-b7b2-38336f2ddee5 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:32.127166 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:32.139821 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cef92-f4a6-4782-818c-68836058dff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:32.140445 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cef92-f4a6-4782-818c-68836058dff7 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:32.140918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cef92-f4a6-4782-818c-68836058dff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:32.141374 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cef92-f4a6-4782-818c-68836058dff7 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:32.142167 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56cef92-f4a6-4782-818c-68836058dff7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:32.144760 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56cef92-f4a6-4782-818c-68836058dff7 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:32.185069 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1331/2650] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:14:31 2022] POST /identity/v3/users => generated 291 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 05:14:32.188845 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10730a-f916-4c5d-8d9c-b1d11f523bd7 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:32.207584 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10730a-f916-4c5d-8d9c-b1d11f523bd7 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:32.213760 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624c420-9b52-4abd-9775-5d05618fe6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:32.214423 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624c420-9b52-4abd-9775-5d05618fe6fc admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:32.225368 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624c420-9b52-4abd-9775-5d05618fe6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:32.226301 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624c420-9b52-4abd-9775-5d05618fe6fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:32.227636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624c420-9b52-4abd-9775-5d05618fe6fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:32.229345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624c420-9b52-4abd-9775-5d05618fe6fc admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:32.245903 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1320/2651] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:14:32.246986 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cef92-f4a6-4782-818c-68836058dff7 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:32.267708 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cef92-f4a6-4782-818c-68836058dff7 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:32.284523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966a0ed-51dc-4b04-8c08-d24007b1a8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:32.285219 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966a0ed-51dc-4b04-8c08-d24007b1a8b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:32.285506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966a0ed-51dc-4b04-8c08-d24007b1a8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:32.285837 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966a0ed-51dc-4b04-8c08-d24007b1a8b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:32.286560 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966a0ed-51dc-4b04-8c08-d24007b1a8b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:32.289207 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966a0ed-51dc-4b04-8c08-d24007b1a8b5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:32.307489 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1332/2652] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:14:32.311936 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624c420-9b52-4abd-9775-5d05618fe6fc admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:32.323556 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624c420-9b52-4abd-9775-5d05618fe6fc admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:32.336894 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896d268-1c87-4903-b0a9-d6b6a73b1f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:32.338084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896d268-1c87-4903-b0a9-d6b6a73b1f4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:32.338370 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896d268-1c87-4903-b0a9-d6b6a73b1f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:32.338701 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896d268-1c87-4903-b0a9-d6b6a73b1f4c admin admin] PATH_INFO: `/v3/projects/0804a3957a284d378026e7b964b00c4f/users/11fa875cd11d4b3695e92777938db8c4/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:32.360090 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896d268-1c87-4903-b0a9-d6b6a73b1f4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0804a3957a284d378026e7b964b00c4f, user_id=11fa875cd11d4b3695e92777938db8c4, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:32.362899 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896d268-1c87-4903-b0a9-d6b6a73b1f4c admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:32.401574 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1321/2653] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:14:32 2022] PUT /identity/v3/projects/0804a3957a284d378026e7b964b00c4f/users/11fa875cd11d4b3695e92777938db8c4/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:32.464517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47311969-7159-4f97-b745-90706a310e06 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:32.464887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47311969-7159-4f97-b745-90706a310e06 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:32.465089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47311969-7159-4f97-b745-90706a310e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:32.505968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-47311969-7159-4f97-b745-90706a310e06 None None] MFA Rules not processed for user `11fa875cd11d4b3695e92777938db8c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:32.507136 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-47311969-7159-4f97-b745-90706a310e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:32.689608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1333/2654] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:32.693043 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896d268-1c87-4903-b0a9-d6b6a73b1f4c admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:32.720068 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:32.814395 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896d268-1c87-4903-b0a9-d6b6a73b1f4c admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:32.823146 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a6426-1dc8-41df-9311-3653287c5d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:32.823850 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a6426-1dc8-41df-9311-3653287c5d49 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:32.824924 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a6426-1dc8-41df-9311-3653287c5d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:32.825212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a6426-1dc8-41df-9311-3653287c5d49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:32.826230 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a6426-1dc8-41df-9311-3653287c5d49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:32.829513 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a6426-1dc8-41df-9311-3653287c5d49 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:32.885750 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1322/2655] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:14:32 2022] POST /identity/v3/projects => generated 383 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:14:32.887257 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47311969-7159-4f97-b745-90706a310e06 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:32.904428 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47311969-7159-4f97-b745-90706a310e06 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:32.999296 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f0605-0a83-4602-b4ff-52adce944553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:33.000127 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f0605-0a83-4602-b4ff-52adce944553 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:33.000522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f0605-0a83-4602-b4ff-52adce944553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:33.000915 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f0605-0a83-4602-b4ff-52adce944553 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:33.001850 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787f0605-0a83-4602-b4ff-52adce944553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:33.004118 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787f0605-0a83-4602-b4ff-52adce944553 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:33.057707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1334/2656] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:14:32 2022] POST /identity/v3/users => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 05:14:33.058594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a6426-1dc8-41df-9311-3653287c5d49 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:33.074545 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a6426-1dc8-41df-9311-3653287c5d49 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:33.080862 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e34045-c414-43d6-af76-4f8d5d02a283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:33.082369 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e34045-c414-43d6-af76-4f8d5d02a283 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:33.082790 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e34045-c414-43d6-af76-4f8d5d02a283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:33.083165 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e34045-c414-43d6-af76-4f8d5d02a283 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:33.085789 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e34045-c414-43d6-af76-4f8d5d02a283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:33.087790 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e34045-c414-43d6-af76-4f8d5d02a283 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:33.100751 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1323/2657] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 05:14:33.105536 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f0605-0a83-4602-b4ff-52adce944553 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:33.118152 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f0605-0a83-4602-b4ff-52adce944553 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:33.128755 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf2c34-f4a2-4b7e-ae8b-b0424dfcc492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:33.128755 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf2c34-f4a2-4b7e-ae8b-b0424dfcc492 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:33.128755 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf2c34-f4a2-4b7e-ae8b-b0424dfcc492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:33.128755 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf2c34-f4a2-4b7e-ae8b-b0424dfcc492 admin admin] PATH_INFO: `/v3/projects/226cc74634fb45ffb63414547b8146fa/users/178bccd05cbc4d719719c07a3a6d3380/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:33.145488 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cf2c34-f4a2-4b7e-ae8b-b0424dfcc492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=226cc74634fb45ffb63414547b8146fa, user_id=178bccd05cbc4d719719c07a3a6d3380, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:33.149636 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cf2c34-f4a2-4b7e-ae8b-b0424dfcc492 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:33.190392 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1335/2658] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:14:33 2022] PUT /identity/v3/projects/226cc74634fb45ffb63414547b8146fa/users/178bccd05cbc4d719719c07a3a6d3380/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 05:14:33.195839 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e34045-c414-43d6-af76-4f8d5d02a283 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:33.208984 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e34045-c414-43d6-af76-4f8d5d02a283 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:33.315573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b4f8e-fff6-4a74-954e-2093bb6cac1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:33.316419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b4f8e-fff6-4a74-954e-2093bb6cac1c admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:33.316803 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b4f8e-fff6-4a74-954e-2093bb6cac1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:33.317175 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b4f8e-fff6-4a74-954e-2093bb6cac1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:33.318158 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b4f8e-fff6-4a74-954e-2093bb6cac1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:33.320803 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b4f8e-fff6-4a74-954e-2093bb6cac1c admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:33.353356 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1324/2659] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 05:14:33.354592 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf2c34-f4a2-4b7e-ae8b-b0424dfcc492 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:33.370957 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf2c34-f4a2-4b7e-ae8b-b0424dfcc492 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:33.384532 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232376af-8a53-4004-915d-e818b3d247e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:33.385529 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232376af-8a53-4004-915d-e818b3d247e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:33.385970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232376af-8a53-4004-915d-e818b3d247e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:33.389755 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232376af-8a53-4004-915d-e818b3d247e6 admin admin] PATH_INFO: `/v3/projects/226cc74634fb45ffb63414547b8146fa/users/178bccd05cbc4d719719c07a3a6d3380/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:33.389755 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232376af-8a53-4004-915d-e818b3d247e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=226cc74634fb45ffb63414547b8146fa, user_id=178bccd05cbc4d719719c07a3a6d3380, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:33.404731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232376af-8a53-4004-915d-e818b3d247e6 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:33.438511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1336/2660] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:14:33 2022] PUT /identity/v3/projects/226cc74634fb45ffb63414547b8146fa/users/178bccd05cbc4d719719c07a3a6d3380/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:33.450745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad82e0-fd77-4d8c-83e6-73518b678d5b None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:33.452362 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad82e0-fd77-4d8c-83e6-73518b678d5b None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:33.453087 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad82e0-fd77-4d8c-83e6-73518b678d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:33.519814 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-93ad82e0-fd77-4d8c-83e6-73518b678d5b None None] MFA Rules not processed for user `178bccd05cbc4d719719c07a3a6d3380`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:33.521148 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-93ad82e0-fd77-4d8c-83e6-73518b678d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:33.676502 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1325/2661] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:33 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:33.685184 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232376af-8a53-4004-915d-e818b3d247e6 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:33.687027 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-232376af-8a53-4004-915d-e818b3d247e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:33.708742 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:33.794500 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232376af-8a53-4004-915d-e818b3d247e6 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:33.798793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ae5cd-52e6-47d7-81da-586569783b12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:33.799542 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ae5cd-52e6-47d7-81da-586569783b12 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:33.799897 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ae5cd-52e6-47d7-81da-586569783b12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:33.800269 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ae5cd-52e6-47d7-81da-586569783b12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:33.815080 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ae5cd-52e6-47d7-81da-586569783b12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:33.816499 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ae5cd-52e6-47d7-81da-586569783b12 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:33.838247 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1337/2662] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:33 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 05:14:33.961096 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad82e0-fd77-4d8c-83e6-73518b678d5b None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:33.971951 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad82e0-fd77-4d8c-83e6-73518b678d5b None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:33.977287 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1545fc06-63dc-4d34-bb31-9d901dc8b517 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:33.978179 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545fc06-63dc-4d34-bb31-9d901dc8b517 service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:33.978536 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545fc06-63dc-4d34-bb31-9d901dc8b517 service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:33.978983 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545fc06-63dc-4d34-bb31-9d901dc8b517 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:33.984130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:34.084351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1545fc06-63dc-4d34-bb31-9d901dc8b517 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:34.086278 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1545fc06-63dc-4d34-bb31-9d901dc8b517 service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:34.113697 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1326/2663] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:33 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:34.962541 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ae5cd-52e6-47d7-81da-586569783b12 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:34.965030 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f53ae5cd-52e6-47d7-81da-586569783b12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:35.067465 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ae5cd-52e6-47d7-81da-586569783b12 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:35.073217 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee785e-6700-4f1c-bea1-6d4aa1bdef22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:35.074063 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee785e-6700-4f1c-bea1-6d4aa1bdef22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:35.074493 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee785e-6700-4f1c-bea1-6d4aa1bdef22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:35.074963 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee785e-6700-4f1c-bea1-6d4aa1bdef22 admin admin] PATH_INFO: `/v3/users/ea11310ddd8249cc93226dd25abaed35` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:35.080712 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee785e-6700-4f1c-bea1-6d4aa1bdef22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea11310ddd8249cc93226dd25abaed35)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:35.083004 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee785e-6700-4f1c-bea1-6d4aa1bdef22 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:35.145123 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-77ee785e-6700-4f1c-bea1-6d4aa1bdef22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea11310ddd8249cc93226dd25abaed35'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:35.160803 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-77ee785e-6700-4f1c-bea1-6d4aa1bdef22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea11310ddd8249cc93226dd25abaed35'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:35.161989 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-77ee785e-6700-4f1c-bea1-6d4aa1bdef22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea11310ddd8249cc93226dd25abaed35'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:35.177765 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-77ee785e-6700-4f1c-bea1-6d4aa1bdef22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea11310ddd8249cc93226dd25abaed35'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:35.178646 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-77ee785e-6700-4f1c-bea1-6d4aa1bdef22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea11310ddd8249cc93226dd25abaed35'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:35.195788 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1338/2664] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:14:34 2022] DELETE /identity/v3/users/ea11310ddd8249cc93226dd25abaed35 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:35.200743 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4b4c3-650f-4445-99b4-2fc8cc9936b9 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:35.201269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4b4c3-650f-4445-99b4-2fc8cc9936b9 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:35.201269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4b4c3-650f-4445-99b4-2fc8cc9936b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:35.230278 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-89a4b4c3-650f-4445-99b4-2fc8cc9936b9 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:35.231591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-89a4b4c3-650f-4445-99b4-2fc8cc9936b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:35.312571 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1327/2665] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 05:14:35.318422 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee785e-6700-4f1c-bea1-6d4aa1bdef22 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:35.320580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-77ee785e-6700-4f1c-bea1-6d4aa1bdef22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:35.373033 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:35.407907 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee785e-6700-4f1c-bea1-6d4aa1bdef22 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:35.412107 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39f675-f88f-428d-bb42-908643f51fdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:35.414054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39f675-f88f-428d-bb42-908643f51fdb service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:35.415268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39f675-f88f-428d-bb42-908643f51fdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:35.415268 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39f675-f88f-428d-bb42-908643f51fdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:35.425723 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39f675-f88f-428d-bb42-908643f51fdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:35.428057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39f675-f88f-428d-bb42-908643f51fdb service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:35.438858 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1339/2666] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:35.950852 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4b4c3-650f-4445-99b4-2fc8cc9936b9 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:35.952424 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-89a4b4c3-650f-4445-99b4-2fc8cc9936b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:35.980897 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4b4c3-650f-4445-99b4-2fc8cc9936b9 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:35.990289 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47feea57-e6e2-4e9f-b883-95960d22173b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:35.992458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47feea57-e6e2-4e9f-b883-95960d22173b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:35.992988 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47feea57-e6e2-4e9f-b883-95960d22173b admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:35.993699 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47feea57-e6e2-4e9f-b883-95960d22173b admin admin] PATH_INFO: `/v3/projects/ee56079a41714a81858f12ec6ef52468` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:35.995235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47feea57-e6e2-4e9f-b883-95960d22173b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee56079a41714a81858f12ec6ef52468)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:35.997875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47feea57-e6e2-4e9f-b883-95960d22173b admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:36.076740 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-47feea57-e6e2-4e9f-b883-95960d22173b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee56079a41714a81858f12ec6ef52468', 'request_id': 'req-47feea57-e6e2-4e9f-b883-95960d22173b'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:36.093568 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-47feea57-e6e2-4e9f-b883-95960d22173b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee56079a41714a81858f12ec6ef52468', 'request_id': 'req-47feea57-e6e2-4e9f-b883-95960d22173b'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:36.097801 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-47feea57-e6e2-4e9f-b883-95960d22173b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee56079a41714a81858f12ec6ef52468', 'request_id': 'req-47feea57-e6e2-4e9f-b883-95960d22173b'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:36.113022 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-47feea57-e6e2-4e9f-b883-95960d22173b admin admin] The token cache is being invalidate because project ee56079a41714a81858f12ec6ef52468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:14:36.113387 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-47feea57-e6e2-4e9f-b883-95960d22173b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:36.114605 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-47feea57-e6e2-4e9f-b883-95960d22173b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:36.119667 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1328/2667] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:14:35 2022] DELETE /identity/v3/projects/ee56079a41714a81858f12ec6ef52468 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:36.121294 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9ec8e-f811-4aa9-8d16-52cd55bd4c09 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:36.121876 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9ec8e-f811-4aa9-8d16-52cd55bd4c09 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:36.121876 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9ec8e-f811-4aa9-8d16-52cd55bd4c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:36.155022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-0ee9ec8e-f811-4aa9-8d16-52cd55bd4c09 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:36.156464 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0ee9ec8e-f811-4aa9-8d16-52cd55bd4c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:36.242203 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1340/2668] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:36.251340 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947d80c-4a4a-4fae-9d39-82ccc588de74 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:36.251944 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947d80c-4a4a-4fae-9d39-82ccc588de74 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:36.251944 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947d80c-4a4a-4fae-9d39-82ccc588de74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:36.278084 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-d947d80c-4a4a-4fae-9d39-82ccc588de74 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:36.279499 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d947d80c-4a4a-4fae-9d39-82ccc588de74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:36.291877 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1329/2669] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 05:14:36.296619 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9ec8e-f811-4aa9-8d16-52cd55bd4c09 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:36.315543 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9ec8e-f811-4aa9-8d16-52cd55bd4c09 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:36.322165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ce2bd-2e87-47e2-b193-047c254f2808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:36.323031 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ce2bd-2e87-47e2-b193-047c254f2808 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:36.323437 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ce2bd-2e87-47e2-b193-047c254f2808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:36.323815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ce2bd-2e87-47e2-b193-047c254f2808 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:36.325003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ce2bd-2e87-47e2-b193-047c254f2808 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 05:14:36.326369 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ce2bd-2e87-47e2-b193-047c254f2808 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:36.328229 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ce2bd-2e87-47e2-b193-047c254f2808 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:36.343664 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1341/2670] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 05:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 05:14:36.347534 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d947d80c-4a4a-4fae-9d39-82ccc588de74 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:36.360272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d947d80c-4a4a-4fae-9d39-82ccc588de74 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:36.365555 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82299b4-24dd-4429-b91a-042e7455401e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:36.366865 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82299b4-24dd-4429-b91a-042e7455401e admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:36.367696 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82299b4-24dd-4429-b91a-042e7455401e admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:36.367696 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82299b4-24dd-4429-b91a-042e7455401e admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:36.368289 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82299b4-24dd-4429-b91a-042e7455401e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:36.370547 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82299b4-24dd-4429-b91a-042e7455401e admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:36.406656 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:36.427198 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1330/2671] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:14:36 2022] POST /identity/v3/projects => generated 385 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:14:36.428784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ce2bd-2e87-47e2-b193-047c254f2808 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:36.445980 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ce2bd-2e87-47e2-b193-047c254f2808 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:36.530097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592607a2-72d1-49b2-bdad-3639422c6075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:36.530718 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592607a2-72d1-49b2-bdad-3639422c6075 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:36.531000 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592607a2-72d1-49b2-bdad-3639422c6075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:36.531274 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592607a2-72d1-49b2-bdad-3639422c6075 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:36.531948 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592607a2-72d1-49b2-bdad-3639422c6075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:36.534364 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592607a2-72d1-49b2-bdad-3639422c6075 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:36.588339 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1342/2672] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:14:36 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 05:14:36.592453 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82299b4-24dd-4429-b91a-042e7455401e admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:36.613278 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82299b4-24dd-4429-b91a-042e7455401e admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:36.643658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af075cd2-dc54-4a55-b0f7-a910648c5560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:36.643658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af075cd2-dc54-4a55-b0f7-a910648c5560 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:36.644449 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af075cd2-dc54-4a55-b0f7-a910648c5560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:36.644905 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af075cd2-dc54-4a55-b0f7-a910648c5560 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:36.648741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af075cd2-dc54-4a55-b0f7-a910648c5560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:36.654194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af075cd2-dc54-4a55-b0f7-a910648c5560 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:36.677300 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1331/2673] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 05:14:36.679834 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592607a2-72d1-49b2-bdad-3639422c6075 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:36.713396 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592607a2-72d1-49b2-bdad-3639422c6075 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:36.720283 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e0540-7a37-453a-8ffe-4a1811070491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:36.720987 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e0540-7a37-453a-8ffe-4a1811070491 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:36.721420 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e0540-7a37-453a-8ffe-4a1811070491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:36.721801 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e0540-7a37-453a-8ffe-4a1811070491 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:36.722587 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e0540-7a37-453a-8ffe-4a1811070491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:36.724766 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e0540-7a37-453a-8ffe-4a1811070491 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:36.747185 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1343/2674] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:14:36.752397 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af075cd2-dc54-4a55-b0f7-a910648c5560 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:36.767624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af075cd2-dc54-4a55-b0f7-a910648c5560 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:36.781848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27437a6b-3e05-49b2-81b8-8582998d7161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:36.782938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27437a6b-3e05-49b2-81b8-8582998d7161 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:36.783577 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27437a6b-3e05-49b2-81b8-8582998d7161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:36.784097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27437a6b-3e05-49b2-81b8-8582998d7161 admin admin] PATH_INFO: `/v3/projects/93c213e8c4fd4c51a625e6c2ccf8bcf7/users/fb67730cd96c4d329af1abf66199e0ca/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:36.831188 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27437a6b-3e05-49b2-81b8-8582998d7161 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93c213e8c4fd4c51a625e6c2ccf8bcf7, user_id=fb67730cd96c4d329af1abf66199e0ca, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:36.833866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27437a6b-3e05-49b2-81b8-8582998d7161 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:36.853997 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1332/2675] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:14:36 2022] PUT /identity/v3/projects/93c213e8c4fd4c51a625e6c2ccf8bcf7/users/fb67730cd96c4d329af1abf66199e0ca/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 05:14:36.864354 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef726b50-cbf2-4915-a4aa-455cdd67ee76 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:36.864797 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef726b50-cbf2-4915-a4aa-455cdd67ee76 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:36.865140 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef726b50-cbf2-4915-a4aa-455cdd67ee76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:36.938021 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ef726b50-cbf2-4915-a4aa-455cdd67ee76 None None] MFA Rules not processed for user `fb67730cd96c4d329af1abf66199e0ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:36.939935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ef726b50-cbf2-4915-a4aa-455cdd67ee76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:37.019559 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:37.091631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1344/2676] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:37.095859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27437a6b-3e05-49b2-81b8-8582998d7161 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:37.203079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27437a6b-3e05-49b2-81b8-8582998d7161 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:37.209211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c4f1c-e14e-4389-8cab-b2b73b773133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:37.210536 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c4f1c-e14e-4389-8cab-b2b73b773133 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:37.210744 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c4f1c-e14e-4389-8cab-b2b73b773133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:37.210913 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c4f1c-e14e-4389-8cab-b2b73b773133 admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:37.211615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c4f1c-e14e-4389-8cab-b2b73b773133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:37.214095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c4f1c-e14e-4389-8cab-b2b73b773133 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:37.240313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc764ff-9b74-4951-ad45-6c583ec779d4 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:37.240900 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc764ff-9b74-4951-ad45-6c583ec779d4 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:37.240900 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc764ff-9b74-4951-ad45-6c583ec779d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:37.254268 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1333/2677] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:14:37 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:14:37.259209 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c4f1c-e14e-4389-8cab-b2b73b773133 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:37.274821 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-fcc764ff-9b74-4951-ad45-6c583ec779d4 None None] MFA Rules not processed for user `28bf7f4d000248c9b40c1b4e596b5d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:37.276483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fcc764ff-9b74-4951-ad45-6c583ec779d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:37.354211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c4f1c-e14e-4389-8cab-b2b73b773133 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:37.358954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8eab20-46f2-4963-b7c8-60ded987fab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:37.359570 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8eab20-46f2-4963-b7c8-60ded987fab5 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:37.359846 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8eab20-46f2-4963-b7c8-60ded987fab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:37.360113 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8eab20-46f2-4963-b7c8-60ded987fab5 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:37.360849 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8eab20-46f2-4963-b7c8-60ded987fab5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:37.363109 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8eab20-46f2-4963-b7c8-60ded987fab5 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:37.368404 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1345/2678] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 05:14:37.386821 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc764ff-9b74-4951-ad45-6c583ec779d4 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:37.389286 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fcc764ff-9b74-4951-ad45-6c583ec779d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:37.400122 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1334/2679] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:14:37 2022] POST /identity/v3/users => generated 292 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 05:14:37.401242 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8eab20-46f2-4963-b7c8-60ded987fab5 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:37.411702 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8eab20-46f2-4963-b7c8-60ded987fab5 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:37.417741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff04345-398c-4a4b-ae80-6d278a2989b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:37.418367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff04345-398c-4a4b-ae80-6d278a2989b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:37.418652 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff04345-398c-4a4b-ae80-6d278a2989b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:37.418928 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff04345-398c-4a4b-ae80-6d278a2989b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:37.420327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff04345-398c-4a4b-ae80-6d278a2989b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:37.421947 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff04345-398c-4a4b-ae80-6d278a2989b9 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:37.434040 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1335/2680] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:14:37.438099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff04345-398c-4a4b-ae80-6d278a2989b9 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:37.448594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff04345-398c-4a4b-ae80-6d278a2989b9 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:37.454131 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9331964-a499-4d27-91db-2e431dfa8f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:37.454790 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9331964-a499-4d27-91db-2e431dfa8f18 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:37.455136 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9331964-a499-4d27-91db-2e431dfa8f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:37.455476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9331964-a499-4d27-91db-2e431dfa8f18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:37.456248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9331964-a499-4d27-91db-2e431dfa8f18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:37.458013 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9331964-a499-4d27-91db-2e431dfa8f18 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:37.473557 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1336/2681] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 05:14:37.474635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9331964-a499-4d27-91db-2e431dfa8f18 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:37.481306 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc764ff-9b74-4951-ad45-6c583ec779d4 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:37.486280 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9331964-a499-4d27-91db-2e431dfa8f18 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:37.487343 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f20c354-9012-4470-8371-0f71d4b15454 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:37.487961 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20c354-9012-4470-8371-0f71d4b15454 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:37.488267 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20c354-9012-4470-8371-0f71d4b15454 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:37.488514 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20c354-9012-4470-8371-0f71d4b15454 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:37.491796 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08339dbc-48cf-4a8d-ac3b-5f955b8153da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:37.492476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08339dbc-48cf-4a8d-ac3b-5f955b8153da admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:37.492751 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08339dbc-48cf-4a8d-ac3b-5f955b8153da admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:37.493020 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08339dbc-48cf-4a8d-ac3b-5f955b8153da admin admin] PATH_INFO: `/v3/projects/621fc54ce675483f8edb0b709095e6c2/users/2ca0b72be13d4365b6ffbe71eea5932e/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:37.498017 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f20c354-9012-4470-8371-0f71d4b15454 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:37.499618 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f20c354-9012-4470-8371-0f71d4b15454 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:37.514076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1346/2682] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:37 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 05:14:37.521436 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08339dbc-48cf-4a8d-ac3b-5f955b8153da admin admin] RBAC: Authorizing `identity:create_grant(project_id=621fc54ce675483f8edb0b709095e6c2, user_id=2ca0b72be13d4365b6ffbe71eea5932e, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:37.524980 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08339dbc-48cf-4a8d-ac3b-5f955b8153da admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:37.527929 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:37.569182 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1337/2683] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:14:37 2022] PUT /identity/v3/projects/621fc54ce675483f8edb0b709095e6c2/users/2ca0b72be13d4365b6ffbe71eea5932e/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 05:14:37.648240 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd4e43-424c-4a1e-aee7-2cf0ff92fb18 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:37.648643 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd4e43-424c-4a1e-aee7-2cf0ff92fb18 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:37.648945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd4e43-424c-4a1e-aee7-2cf0ff92fb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:37.686606 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-08dd4e43-424c-4a1e-aee7-2cf0ff92fb18 None None] MFA Rules not processed for user `2ca0b72be13d4365b6ffbe71eea5932e`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:37.687925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-08dd4e43-424c-4a1e-aee7-2cf0ff92fb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:37.741332 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:37.789590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1347/2684] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:37.790726 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08339dbc-48cf-4a8d-ac3b-5f955b8153da admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:37.875442 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08339dbc-48cf-4a8d-ac3b-5f955b8153da admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:37.883900 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4150b05-797e-495f-9aee-4dbba7c4a2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:37.884574 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4150b05-797e-495f-9aee-4dbba7c4a2d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:37.884995 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4150b05-797e-495f-9aee-4dbba7c4a2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:37.885275 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4150b05-797e-495f-9aee-4dbba7c4a2d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:37.886208 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4150b05-797e-495f-9aee-4dbba7c4a2d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:37.888458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4150b05-797e-495f-9aee-4dbba7c4a2d1 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:37.932390 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1338/2685] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:14:37 2022] POST /identity/v3/projects => generated 383 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:14:37.936935 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd4e43-424c-4a1e-aee7-2cf0ff92fb18 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:37.954889 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd4e43-424c-4a1e-aee7-2cf0ff92fb18 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:38.031450 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7c1f0-fc86-40f2-b709-4cc71417739d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:38.032091 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7c1f0-fc86-40f2-b709-4cc71417739d admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:38.033017 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7c1f0-fc86-40f2-b709-4cc71417739d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:38.033017 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7c1f0-fc86-40f2-b709-4cc71417739d admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:38.033382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a7c1f0-fc86-40f2-b709-4cc71417739d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:38.035337 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a7c1f0-fc86-40f2-b709-4cc71417739d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:38.070664 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1348/2686] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:14:37 2022] POST /identity/v3/users => generated 298 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:14:38.071107 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4150b05-797e-495f-9aee-4dbba7c4a2d1 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:38.086218 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4150b05-797e-495f-9aee-4dbba7c4a2d1 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:38.091515 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aac885-7a76-4e98-a1fd-b89632337be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:38.092226 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aac885-7a76-4e98-a1fd-b89632337be1 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:38.092595 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aac885-7a76-4e98-a1fd-b89632337be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:38.092864 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aac885-7a76-4e98-a1fd-b89632337be1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:38.093671 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aac885-7a76-4e98-a1fd-b89632337be1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:38.095990 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7c1f0-fc86-40f2-b709-4cc71417739d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:38.096820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aac885-7a76-4e98-a1fd-b89632337be1 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:38.098062 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f4a7c1f0-fc86-40f2-b709-4cc71417739d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:38.110329 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1339/2687] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 05:14:38.111766 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7c1f0-fc86-40f2-b709-4cc71417739d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:38.114313 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aac885-7a76-4e98-a1fd-b89632337be1 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:38.117343 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04ad26-fad1-497c-aee3-4b40a4f5a0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:38.118029 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04ad26-fad1-497c-aee3-4b40a4f5a0ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:38.118306 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04ad26-fad1-497c-aee3-4b40a4f5a0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:38.118709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04ad26-fad1-497c-aee3-4b40a4f5a0ee admin admin] PATH_INFO: `/v3/users/11fa875cd11d4b3695e92777938db8c4` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:38.120101 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04ad26-fad1-497c-aee3-4b40a4f5a0ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=11fa875cd11d4b3695e92777938db8c4)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:38.122077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04ad26-fad1-497c-aee3-4b40a4f5a0ee admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:38.125946 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aac885-7a76-4e98-a1fd-b89632337be1 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:38.131441 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d121999-e8bb-431c-83b7-539902665889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:38.132685 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d121999-e8bb-431c-83b7-539902665889 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:38.132968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d121999-e8bb-431c-83b7-539902665889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:38.133249 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d121999-e8bb-431c-83b7-539902665889 admin admin] PATH_INFO: `/v3/projects/1aec933a744649b6b32d013e946f40a5/users/b9a0c47eca6d4e73ab3f0d691e653d7b/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:38.152315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d121999-e8bb-431c-83b7-539902665889 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aec933a744649b6b32d013e946f40a5, user_id=b9a0c47eca6d4e73ab3f0d691e653d7b, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:38.156018 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d121999-e8bb-431c-83b7-539902665889 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:38.186242 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1340/2688] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:14:38 2022] PUT /identity/v3/projects/1aec933a744649b6b32d013e946f40a5/users/b9a0c47eca6d4e73ab3f0d691e653d7b/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 05:14:38.190334 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d121999-e8bb-431c-83b7-539902665889 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:38.192161 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-9b04ad26-fad1-497c-aee3-4b40a4f5a0ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11fa875cd11d4b3695e92777938db8c4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:38.200937 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-9b04ad26-fad1-497c-aee3-4b40a4f5a0ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11fa875cd11d4b3695e92777938db8c4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:38.201819 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-9b04ad26-fad1-497c-aee3-4b40a4f5a0ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11fa875cd11d4b3695e92777938db8c4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:38.218411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-9b04ad26-fad1-497c-aee3-4b40a4f5a0ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11fa875cd11d4b3695e92777938db8c4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:38.219790 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-9b04ad26-fad1-497c-aee3-4b40a4f5a0ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11fa875cd11d4b3695e92777938db8c4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:38.240197 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1349/2689] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:14:38 2022] DELETE /identity/v3/users/11fa875cd11d4b3695e92777938db8c4 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 05:14:38.245454 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae8b46-8c70-467d-859b-7750c7cc54e8 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:38.246142 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae8b46-8c70-467d-859b-7750c7cc54e8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:38.246142 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae8b46-8c70-467d-859b-7750c7cc54e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:38.268603 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-74ae8b46-8c70-467d-859b-7750c7cc54e8 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:38.269316 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d121999-e8bb-431c-83b7-539902665889 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:38.270106 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-74ae8b46-8c70-467d-859b-7750c7cc54e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:38.316325 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:38.326583 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:38.342591 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b3c4a-32c7-4818-a105-7597bbcae72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:38.343613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b3c4a-32c7-4818-a105-7597bbcae72f admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:38.343613 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b3c4a-32c7-4818-a105-7597bbcae72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:38.343938 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b3c4a-32c7-4818-a105-7597bbcae72f admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:38.344709 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224b3c4a-32c7-4818-a105-7597bbcae72f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:38.346511 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224b3c4a-32c7-4818-a105-7597bbcae72f admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:38.354102 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1350/2690] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 05:14:38.360635 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae8b46-8c70-467d-859b-7750c7cc54e8 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:38.361984 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1341/2691] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 05:14:38.367010 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-74ae8b46-8c70-467d-859b-7750c7cc54e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:38.367604 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b3c4a-32c7-4818-a105-7597bbcae72f admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:38.369612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-224b3c4a-32c7-4818-a105-7597bbcae72f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:38.382644 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b3c4a-32c7-4818-a105-7597bbcae72f admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:38.390038 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92cfccc-4bc0-405f-9217-59d5f2ccb533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:38.391209 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cfccc-4bc0-405f-9217-59d5f2ccb533 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:38.391535 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cfccc-4bc0-405f-9217-59d5f2ccb533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:38.392336 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cfccc-4bc0-405f-9217-59d5f2ccb533 admin admin] PATH_INFO: `/v3/projects/1aec933a744649b6b32d013e946f40a5/users/b9a0c47eca6d4e73ab3f0d691e653d7b/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:38.394939 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92cfccc-4bc0-405f-9217-59d5f2ccb533 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aec933a744649b6b32d013e946f40a5, user_id=b9a0c47eca6d4e73ab3f0d691e653d7b, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:38.398334 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92cfccc-4bc0-405f-9217-59d5f2ccb533 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:38.409514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1342/2692] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:14:38 2022] PUT /identity/v3/projects/1aec933a744649b6b32d013e946f40a5/users/b9a0c47eca6d4e73ab3f0d691e653d7b/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:38.422824 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3510c2c-7e54-46ef-ab25-61eb07c93d43 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:38.423331 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3510c2c-7e54-46ef-ab25-61eb07c93d43 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:38.423511 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3510c2c-7e54-46ef-ab25-61eb07c93d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:38.445656 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae8b46-8c70-467d-859b-7750c7cc54e8 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:38.455649 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-f3510c2c-7e54-46ef-ab25-61eb07c93d43 None None] MFA Rules not processed for user `b9a0c47eca6d4e73ab3f0d691e653d7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:38.457016 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-f3510c2c-7e54-46ef-ab25-61eb07c93d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:38.513787 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2226cb2-cbf3-4a9a-a0aa-b9f83ec76b2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:38.514470 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2226cb2-cbf3-4a9a-a0aa-b9f83ec76b2c service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:38.514766 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2226cb2-cbf3-4a9a-a0aa-b9f83ec76b2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:38.515046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2226cb2-cbf3-4a9a-a0aa-b9f83ec76b2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:38.546150 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1343/2693] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:38 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 05:14:38.558416 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3510c2c-7e54-46ef-ab25-61eb07c93d43 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:38.560131 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-f3510c2c-7e54-46ef-ab25-61eb07c93d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:38.603849 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2226cb2-cbf3-4a9a-a0aa-b9f83ec76b2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:38.609581 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2226cb2-cbf3-4a9a-a0aa-b9f83ec76b2c service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:38.619754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1351/2694] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 05:14:38.641131 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3510c2c-7e54-46ef-ab25-61eb07c93d43 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:38.644624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdff9ae-c32d-4b06-89c0-630d87df7ccc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:38.645290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdff9ae-c32d-4b06-89c0-630d87df7ccc service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:38.645579 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdff9ae-c32d-4b06-89c0-630d87df7ccc service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:38.645865 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdff9ae-c32d-4b06-89c0-630d87df7ccc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:38.648252 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-fbdff9ae-c32d-4b06-89c0-630d87df7ccc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:38.741741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdff9ae-c32d-4b06-89c0-630d87df7ccc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:38.745059 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdff9ae-c32d-4b06-89c0-630d87df7ccc service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:38.772913 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1344/2695] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 05:14:39.071495 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2226cb2-cbf3-4a9a-a0aa-b9f83ec76b2c service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:39.080406 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e2226cb2-cbf3-4a9a-a0aa-b9f83ec76b2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:39.102248 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2226cb2-cbf3-4a9a-a0aa-b9f83ec76b2c service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:39.108730 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447af306-b568-4bce-bf97-9735bcb3526e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:39.109629 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447af306-b568-4bce-bf97-9735bcb3526e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:39.110011 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447af306-b568-4bce-bf97-9735bcb3526e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:39.110414 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447af306-b568-4bce-bf97-9735bcb3526e admin admin] PATH_INFO: `/v3/projects/0804a3957a284d378026e7b964b00c4f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:39.112253 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447af306-b568-4bce-bf97-9735bcb3526e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0804a3957a284d378026e7b964b00c4f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:39.114752 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447af306-b568-4bce-bf97-9735bcb3526e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:39.192933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:39.226374 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-447af306-b568-4bce-bf97-9735bcb3526e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0804a3957a284d378026e7b964b00c4f', 'request_id': 'req-447af306-b568-4bce-bf97-9735bcb3526e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:39.243566 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-447af306-b568-4bce-bf97-9735bcb3526e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0804a3957a284d378026e7b964b00c4f', 'request_id': 'req-447af306-b568-4bce-bf97-9735bcb3526e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:39.250991 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-447af306-b568-4bce-bf97-9735bcb3526e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0804a3957a284d378026e7b964b00c4f', 'request_id': 'req-447af306-b568-4bce-bf97-9735bcb3526e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:39.251621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52440fc3-412b-4ed5-b1cb-451f75e36cde None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:39.251806 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52440fc3-412b-4ed5-b1cb-451f75e36cde None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:39.252092 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52440fc3-412b-4ed5-b1cb-451f75e36cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:39.266855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-447af306-b568-4bce-bf97-9735bcb3526e admin admin] The token cache is being invalidate because project 0804a3957a284d378026e7b964b00c4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:14:39.267147 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-447af306-b568-4bce-bf97-9735bcb3526e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:39.268884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-447af306-b568-4bce-bf97-9735bcb3526e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:39.270937 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1352/2696] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:14:39 2022] DELETE /identity/v3/projects/0804a3957a284d378026e7b964b00c4f => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:39.276160 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447af306-b568-4bce-bf97-9735bcb3526e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:39.282349 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-447af306-b568-4bce-bf97-9735bcb3526e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:39.299376 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-52440fc3-412b-4ed5-b1cb-451f75e36cde None None] MFA Rules not processed for user `28bf7f4d000248c9b40c1b4e596b5d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:39.300770 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-52440fc3-412b-4ed5-b1cb-451f75e36cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:39.313971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:39.380322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447af306-b568-4bce-bf97-9735bcb3526e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:39.385486 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829282e-c1e1-483b-bdd6-fbb2e59dc32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:39.386203 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829282e-c1e1-483b-bdd6-fbb2e59dc32e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:39.386484 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829282e-c1e1-483b-bdd6-fbb2e59dc32e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:39.386767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829282e-c1e1-483b-bdd6-fbb2e59dc32e admin admin] PATH_INFO: `/v3/users/178bccd05cbc4d719719c07a3a6d3380` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:39.387987 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829282e-c1e1-483b-bdd6-fbb2e59dc32e admin admin] RBAC: Authorizing `identity:delete_user(user_id=178bccd05cbc4d719719c07a3a6d3380)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:39.390070 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829282e-c1e1-483b-bdd6-fbb2e59dc32e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:39.398637 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1345/2697] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:39 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:39.447880 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b829282e-c1e1-483b-bdd6-fbb2e59dc32e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '178bccd05cbc4d719719c07a3a6d3380'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:39.456295 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b829282e-c1e1-483b-bdd6-fbb2e59dc32e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '178bccd05cbc4d719719c07a3a6d3380'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:39.457160 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b829282e-c1e1-483b-bdd6-fbb2e59dc32e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '178bccd05cbc4d719719c07a3a6d3380'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:39.471734 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b829282e-c1e1-483b-bdd6-fbb2e59dc32e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '178bccd05cbc4d719719c07a3a6d3380'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:39.472508 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b829282e-c1e1-483b-bdd6-fbb2e59dc32e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '178bccd05cbc4d719719c07a3a6d3380'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:39.488432 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1353/2698] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:14:39 2022] DELETE /identity/v3/users/178bccd05cbc4d719719c07a3a6d3380 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:39.828205 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52440fc3-412b-4ed5-b1cb-451f75e36cde None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:39.829768 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-52440fc3-412b-4ed5-b1cb-451f75e36cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:39.888508 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829282e-c1e1-483b-bdd6-fbb2e59dc32e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:39.890263 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b829282e-c1e1-483b-bdd6-fbb2e59dc32e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:39.902365 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52440fc3-412b-4ed5-b1cb-451f75e36cde None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:39.956478 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7749c16-1ce9-45df-9edf-79b498663d87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:39.957670 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7749c16-1ce9-45df-9edf-79b498663d87 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:39.957832 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:39.957967 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7749c16-1ce9-45df-9edf-79b498663d87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:39.958217 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7749c16-1ce9-45df-9edf-79b498663d87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:39.960088 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d7749c16-1ce9-45df-9edf-79b498663d87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:39.970791 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7749c16-1ce9-45df-9edf-79b498663d87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:39.972257 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7749c16-1ce9-45df-9edf-79b498663d87 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:39.972555 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829282e-c1e1-483b-bdd6-fbb2e59dc32e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:39.978510 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137269b5-95fe-429f-9f1c-b8ae79be4d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:39.979159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137269b5-95fe-429f-9f1c-b8ae79be4d2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:39.979439 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137269b5-95fe-429f-9f1c-b8ae79be4d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:39.979715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137269b5-95fe-429f-9f1c-b8ae79be4d2d admin admin] PATH_INFO: `/v3/projects/226cc74634fb45ffb63414547b8146fa` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:39.980870 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137269b5-95fe-429f-9f1c-b8ae79be4d2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=226cc74634fb45ffb63414547b8146fa)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:39.981287 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1346/2699] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:39.983004 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137269b5-95fe-429f-9f1c-b8ae79be4d2d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:40.032333 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-137269b5-95fe-429f-9f1c-b8ae79be4d2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '226cc74634fb45ffb63414547b8146fa', 'request_id': 'req-137269b5-95fe-429f-9f1c-b8ae79be4d2d'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:40.042478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-137269b5-95fe-429f-9f1c-b8ae79be4d2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '226cc74634fb45ffb63414547b8146fa', 'request_id': 'req-137269b5-95fe-429f-9f1c-b8ae79be4d2d'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:40.048543 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-137269b5-95fe-429f-9f1c-b8ae79be4d2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '226cc74634fb45ffb63414547b8146fa', 'request_id': 'req-137269b5-95fe-429f-9f1c-b8ae79be4d2d'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:40.060931 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-137269b5-95fe-429f-9f1c-b8ae79be4d2d admin admin] The token cache is being invalidate because project 226cc74634fb45ffb63414547b8146fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:14:40.061221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-137269b5-95fe-429f-9f1c-b8ae79be4d2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:40.062002 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-137269b5-95fe-429f-9f1c-b8ae79be4d2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:40.063632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1354/2700] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:14:39 2022] DELETE /identity/v3/projects/226cc74634fb45ffb63414547b8146fa => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:40.069394 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7a997-08d5-487a-bca6-26716ef16237 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:40.069706 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7a997-08d5-487a-bca6-26716ef16237 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:40.070099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7a997-08d5-487a-bca6-26716ef16237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:40.090595 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-5cc7a997-08d5-487a-bca6-26716ef16237 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:40.091800 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5cc7a997-08d5-487a-bca6-26716ef16237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:40.169005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1347/2701] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:40.177770 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c89711-f4cb-4a82-bed3-08117750ff8e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:40.178059 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c89711-f4cb-4a82-bed3-08117750ff8e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:40.178367 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c89711-f4cb-4a82-bed3-08117750ff8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:40.201198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-23c89711-f4cb-4a82-bed3-08117750ff8e None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:40.202447 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-23c89711-f4cb-4a82-bed3-08117750ff8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:40.215149 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1355/2702] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:40.216357 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7a997-08d5-487a-bca6-26716ef16237 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:40.236699 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7a997-08d5-487a-bca6-26716ef16237 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:40.243830 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e501f-da16-47f0-9938-92303112de7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:40.244620 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e501f-da16-47f0-9938-92303112de7b admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:40.244914 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e501f-da16-47f0-9938-92303112de7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:40.245186 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e501f-da16-47f0-9938-92303112de7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:40.245923 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e501f-da16-47f0-9938-92303112de7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 05:14:40.246226 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e501f-da16-47f0-9938-92303112de7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:40.247815 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e501f-da16-47f0-9938-92303112de7b admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:40.265219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1348/2703] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 05:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 05:14:40.269938 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c89711-f4cb-4a82-bed3-08117750ff8e None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:40.281996 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c89711-f4cb-4a82-bed3-08117750ff8e None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:40.287091 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678dce12-51f4-4403-a2e2-c96551e01a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:40.287603 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678dce12-51f4-4403-a2e2-c96551e01a06 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:40.288154 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678dce12-51f4-4403-a2e2-c96551e01a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:40.288430 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678dce12-51f4-4403-a2e2-c96551e01a06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:40.289489 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678dce12-51f4-4403-a2e2-c96551e01a06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:40.291804 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678dce12-51f4-4403-a2e2-c96551e01a06 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:40.326620 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1356/2704] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:14:40 2022] POST /identity/v3/projects => generated 359 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:14:40.330926 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e501f-da16-47f0-9938-92303112de7b admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:40.342825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:40.358918 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e501f-da16-47f0-9938-92303112de7b admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:40.394791 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678dce12-51f4-4403-a2e2-c96551e01a06 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:40.396943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-678dce12-51f4-4403-a2e2-c96551e01a06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:40.499857 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10894436-298c-48df-af74-8dea01e35658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:40.500492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10894436-298c-48df-af74-8dea01e35658 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:40.500776 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10894436-298c-48df-af74-8dea01e35658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:40.501128 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10894436-298c-48df-af74-8dea01e35658 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:40.501793 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10894436-298c-48df-af74-8dea01e35658 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:40.503799 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10894436-298c-48df-af74-8dea01e35658 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:40.537618 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1349/2705] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:14:40 2022] POST /identity/v3/users => generated 280 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 05:14:40.541417 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10894436-298c-48df-af74-8dea01e35658 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:40.552035 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10894436-298c-48df-af74-8dea01e35658 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:40.556623 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678dce12-51f4-4403-a2e2-c96551e01a06 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:40.558246 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895374a-af0a-46cd-95ad-6a11532d19b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:40.558937 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895374a-af0a-46cd-95ad-6a11532d19b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:40.559218 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895374a-af0a-46cd-95ad-6a11532d19b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:40.559492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895374a-af0a-46cd-95ad-6a11532d19b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:40.560216 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895374a-af0a-46cd-95ad-6a11532d19b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:40.561796 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895374a-af0a-46cd-95ad-6a11532d19b5 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:40.563783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea84b33-243d-45f8-bf41-61704288fbd9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350f72f740f749f68175c1c4e8ae9d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:40.564843 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea84b33-243d-45f8-bf41-61704288fbd9 service placement] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:40.565241 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea84b33-243d-45f8-bf41-61704288fbd9 service placement] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:40.565521 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea84b33-243d-45f8-bf41-61704288fbd9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:40.567430 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-dea84b33-243d-45f8-bf41-61704288fbd9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:40.575034 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1350/2706] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:14:40.582844 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895374a-af0a-46cd-95ad-6a11532d19b5 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:40.588005 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea84b33-243d-45f8-bf41-61704288fbd9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:40.589653 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea84b33-243d-45f8-bf41-61704288fbd9 service placement] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:40.594902 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895374a-af0a-46cd-95ad-6a11532d19b5 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:40.601738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1357/2707] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 05:14:40.615504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847a02a-93fa-460b-87e3-b4e99ca71c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:40.615504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847a02a-93fa-460b-87e3-b4e99ca71c23 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:40.615504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847a02a-93fa-460b-87e3-b4e99ca71c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:40.615504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847a02a-93fa-460b-87e3-b4e99ca71c23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:40.616568 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847a02a-93fa-460b-87e3-b4e99ca71c23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:40.620029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847a02a-93fa-460b-87e3-b4e99ca71c23 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:40.635640 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1351/2708] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:14:40.642154 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea84b33-243d-45f8-bf41-61704288fbd9 service placement] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:40.659310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea84b33-243d-45f8-bf41-61704288fbd9 service placement] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:40.662278 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847a02a-93fa-460b-87e3-b4e99ca71c23 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:40.665440 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d1495-6492-4d37-b41c-d27c8cf8db38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:40.666146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d1495-6492-4d37-b41c-d27c8cf8db38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:40.666424 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d1495-6492-4d37-b41c-d27c8cf8db38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:40.666699 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d1495-6492-4d37-b41c-d27c8cf8db38 admin admin] PATH_INFO: `/v3/projects/0517ffc15965481baa5820f7437aff20/users/efe77c4809d24c6395aa6a70bdb8f77c/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:40.666958 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-3847a02a-93fa-460b-87e3-b4e99ca71c23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:40.685351 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91d1495-6492-4d37-b41c-d27c8cf8db38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0517ffc15965481baa5820f7437aff20, user_id=efe77c4809d24c6395aa6a70bdb8f77c, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:40.688489 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91d1495-6492-4d37-b41c-d27c8cf8db38 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:40.724594 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1358/2709] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:14:40 2022] PUT /identity/v3/projects/0517ffc15965481baa5820f7437aff20/users/efe77c4809d24c6395aa6a70bdb8f77c/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:40.736016 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e51f77-c17b-4077-ae9b-4f896db188df None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:40.736441 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e51f77-c17b-4077-ae9b-4f896db188df None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:40.736557 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e51f77-c17b-4077-ae9b-4f896db188df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:40.772104 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b8e51f77-c17b-4077-ae9b-4f896db188df None None] MFA Rules not processed for user `efe77c4809d24c6395aa6a70bdb8f77c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:40.776191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b8e51f77-c17b-4077-ae9b-4f896db188df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:40.781287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:40.796314 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847a02a-93fa-460b-87e3-b4e99ca71c23 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:40.906180 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f33242-624b-496d-a2ab-ff4550a652cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:40.907142 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f33242-624b-496d-a2ab-ff4550a652cb service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:40.907436 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f33242-624b-496d-a2ab-ff4550a652cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:40.907838 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f33242-624b-496d-a2ab-ff4550a652cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:40.909974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c0f33242-624b-496d-a2ab-ff4550a652cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:40.945433 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1359/2710] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:40 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 05:14:40.987316 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e51f77-c17b-4077-ae9b-4f896db188df None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:40.989403 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b8e51f77-c17b-4077-ae9b-4f896db188df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:41.006316 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e51f77-c17b-4077-ae9b-4f896db188df None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:41.012322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33024274-d9e7-4975-a61a-1331510e24a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:41.013070 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33024274-d9e7-4975-a61a-1331510e24a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:41.013331 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33024274-d9e7-4975-a61a-1331510e24a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:41.013719 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33024274-d9e7-4975-a61a-1331510e24a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:41.014425 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:41.025405 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33024274-d9e7-4975-a61a-1331510e24a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:41.027880 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33024274-d9e7-4975-a61a-1331510e24a0 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:41.063694 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1360/2711] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:40 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 05:14:41.067143 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f33242-624b-496d-a2ab-ff4550a652cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:41.070919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f33242-624b-496d-a2ab-ff4550a652cb service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:41.073155 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33024274-d9e7-4975-a61a-1331510e24a0 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:41.095588 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1352/2712] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:41.175175 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33024274-d9e7-4975-a61a-1331510e24a0 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:41.191419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d32fe-e74c-46f3-8321-926174a10b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:41.192149 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d32fe-e74c-46f3-8321-926174a10b8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:41.197563 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d32fe-e74c-46f3-8321-926174a10b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:41.197943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d32fe-e74c-46f3-8321-926174a10b8e admin admin] PATH_INFO: `/v3/users/efe77c4809d24c6395aa6a70bdb8f77c` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:41.202400 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657d32fe-e74c-46f3-8321-926174a10b8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=efe77c4809d24c6395aa6a70bdb8f77c)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:41.205771 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657d32fe-e74c-46f3-8321-926174a10b8e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:41.302183 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-657d32fe-e74c-46f3-8321-926174a10b8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efe77c4809d24c6395aa6a70bdb8f77c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:41.313856 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-657d32fe-e74c-46f3-8321-926174a10b8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efe77c4809d24c6395aa6a70bdb8f77c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:41.314638 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-657d32fe-e74c-46f3-8321-926174a10b8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efe77c4809d24c6395aa6a70bdb8f77c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:41.331031 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-657d32fe-e74c-46f3-8321-926174a10b8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efe77c4809d24c6395aa6a70bdb8f77c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:41.334099 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-657d32fe-e74c-46f3-8321-926174a10b8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efe77c4809d24c6395aa6a70bdb8f77c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:41.356143 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1361/2713] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:14:41 2022] DELETE /identity/v3/users/efe77c4809d24c6395aa6a70bdb8f77c => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:41.361244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ac796-b0d9-45d6-9576-9dc724471fb6 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:41.361781 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ac796-b0d9-45d6-9576-9dc724471fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:41.361903 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ac796-b0d9-45d6-9576-9dc724471fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:41.390864 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-377ac796-b0d9-45d6-9576-9dc724471fb6 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:41.392271 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-377ac796-b0d9-45d6-9576-9dc724471fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:41.466920 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1353/2714] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:41.480986 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d32fe-e74c-46f3-8321-926174a10b8e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:41.482587 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-657d32fe-e74c-46f3-8321-926174a10b8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:41.499415 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:41.602704 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d32fe-e74c-46f3-8321-926174a10b8e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:41.607085 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2c144-2c37-48ca-bb22-cadb0affe781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:41.607964 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2c144-2c37-48ca-bb22-cadb0affe781 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:41.608376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2c144-2c37-48ca-bb22-cadb0affe781 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:41.608752 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2c144-2c37-48ca-bb22-cadb0affe781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:41.634506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2c144-2c37-48ca-bb22-cadb0affe781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:41.636748 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2c144-2c37-48ca-bb22-cadb0affe781 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:41.652942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1362/2715] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 05:14:41.966562 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ac796-b0d9-45d6-9576-9dc724471fb6 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:41.968058 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-377ac796-b0d9-45d6-9576-9dc724471fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:42.078944 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ac796-b0d9-45d6-9576-9dc724471fb6 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:42.085628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c698d9-03b3-4000-b162-ba7b570e84f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:42.086442 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c698d9-03b3-4000-b162-ba7b570e84f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:42.086826 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c698d9-03b3-4000-b162-ba7b570e84f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:42.087225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c698d9-03b3-4000-b162-ba7b570e84f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:42.089413 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d3c698d9-03b3-4000-b162-ba7b570e84f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:42.094353 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2c144-2c37-48ca-bb22-cadb0affe781 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:42.095889 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-12a2c144-2c37-48ca-bb22-cadb0affe781 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:42.115534 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2c144-2c37-48ca-bb22-cadb0affe781 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:42.123346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee3370d-0314-4a84-ae1f-8e93384c8656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:42.124278 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3370d-0314-4a84-ae1f-8e93384c8656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:42.124669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3370d-0314-4a84-ae1f-8e93384c8656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:42.125170 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3370d-0314-4a84-ae1f-8e93384c8656 admin admin] PATH_INFO: `/v3/projects/0517ffc15965481baa5820f7437aff20` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:42.126787 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee3370d-0314-4a84-ae1f-8e93384c8656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0517ffc15965481baa5820f7437aff20)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:42.128870 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee3370d-0314-4a84-ae1f-8e93384c8656 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:42.201224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c698d9-03b3-4000-b162-ba7b570e84f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:42.202787 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c698d9-03b3-4000-b162-ba7b570e84f8 service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:42.219735 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4ee3370d-0314-4a84-ae1f-8e93384c8656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0517ffc15965481baa5820f7437aff20', 'request_id': 'req-4ee3370d-0314-4a84-ae1f-8e93384c8656'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:42.231200 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1354/2716] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:41 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:42.257584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4ee3370d-0314-4a84-ae1f-8e93384c8656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0517ffc15965481baa5820f7437aff20', 'request_id': 'req-4ee3370d-0314-4a84-ae1f-8e93384c8656'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:42.266449 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4ee3370d-0314-4a84-ae1f-8e93384c8656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0517ffc15965481baa5820f7437aff20', 'request_id': 'req-4ee3370d-0314-4a84-ae1f-8e93384c8656'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:42.280828 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4ee3370d-0314-4a84-ae1f-8e93384c8656 admin admin] The token cache is being invalidate because project 0517ffc15965481baa5820f7437aff20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:14:42.281418 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4ee3370d-0314-4a84-ae1f-8e93384c8656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:42.283323 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4ee3370d-0314-4a84-ae1f-8e93384c8656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:42.286292 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1363/2717] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:14:42 2022] DELETE /identity/v3/projects/0517ffc15965481baa5820f7437aff20 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:42.293968 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1117d-362c-4a73-b28e-14e857912831 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:42.294196 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1117d-362c-4a73-b28e-14e857912831 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:42.294506 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1117d-362c-4a73-b28e-14e857912831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:42.341180 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-24d1117d-362c-4a73-b28e-14e857912831 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:42.342498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-24d1117d-362c-4a73-b28e-14e857912831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:42.426654 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:42.474186 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1355/2718] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:42.484471 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7b91a-cf45-4b9a-886d-f2d574119843 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:42.484471 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7b91a-cf45-4b9a-886d-f2d574119843 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:42.484471 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7b91a-cf45-4b9a-886d-f2d574119843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:42.508495 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-d1a7b91a-cf45-4b9a-886d-f2d574119843 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:42.509564 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d1a7b91a-cf45-4b9a-886d-f2d574119843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:42.524821 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1364/2719] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:42.529067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1117d-362c-4a73-b28e-14e857912831 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:42.542016 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1117d-362c-4a73-b28e-14e857912831 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:42.547388 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ad003-134d-4d3f-a3ac-fb50828bb0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:42.548166 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ad003-134d-4d3f-a3ac-fb50828bb0e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:42.548618 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ad003-134d-4d3f-a3ac-fb50828bb0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:42.548992 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ad003-134d-4d3f-a3ac-fb50828bb0e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:42.549859 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ad003-134d-4d3f-a3ac-fb50828bb0e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 05:14:42.551335 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ad003-134d-4d3f-a3ac-fb50828bb0e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:42.553782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ad003-134d-4d3f-a3ac-fb50828bb0e3 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:42.571061 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1356/2720] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 05:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 05:14:42.576283 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7b91a-cf45-4b9a-886d-f2d574119843 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:42.610339 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7b91a-cf45-4b9a-886d-f2d574119843 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:42.618019 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c308bf8-93b2-478c-9a2a-631dff830dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:42.618547 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c308bf8-93b2-478c-9a2a-631dff830dcd admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:42.619109 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c308bf8-93b2-478c-9a2a-631dff830dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:42.619310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c308bf8-93b2-478c-9a2a-631dff830dcd admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:42.620564 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c308bf8-93b2-478c-9a2a-631dff830dcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:42.632502 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c308bf8-93b2-478c-9a2a-631dff830dcd admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:42.691082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1365/2721] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:14:42 2022] POST /identity/v3/projects => generated 371 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:14:42.695903 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ad003-134d-4d3f-a3ac-fb50828bb0e3 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:42.709026 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ad003-134d-4d3f-a3ac-fb50828bb0e3 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:42.809023 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0dc95f-c34f-484f-9520-10a11cb1fab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:42.809996 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0dc95f-c34f-484f-9520-10a11cb1fab9 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:42.810389 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0dc95f-c34f-484f-9520-10a11cb1fab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:42.810732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0dc95f-c34f-484f-9520-10a11cb1fab9 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:42.811643 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0dc95f-c34f-484f-9520-10a11cb1fab9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:42.814193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0dc95f-c34f-484f-9520-10a11cb1fab9 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:42.857658 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1357/2722] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:14:42 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 05:14:42.861595 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c308bf8-93b2-478c-9a2a-631dff830dcd admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:42.881158 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c308bf8-93b2-478c-9a2a-631dff830dcd admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:42.887853 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b0e86-206d-42e8-90d5-2cc141d05e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:42.888788 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b0e86-206d-42e8-90d5-2cc141d05e03 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:42.889373 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b0e86-206d-42e8-90d5-2cc141d05e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:42.889911 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b0e86-206d-42e8-90d5-2cc141d05e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:42.890830 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b0e86-206d-42e8-90d5-2cc141d05e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:42.893158 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b0e86-206d-42e8-90d5-2cc141d05e03 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:42.918595 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0dc95f-c34f-484f-9520-10a11cb1fab9 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:42.919181 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1366/2723] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:14:42.929969 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0dc95f-c34f-484f-9520-10a11cb1fab9 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:42.935077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e25b10-9275-4ff9-bf20-4091b26fc01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:42.935837 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e25b10-9275-4ff9-bf20-4091b26fc01d admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:42.936245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e25b10-9275-4ff9-bf20-4091b26fc01d admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:42.936626 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e25b10-9275-4ff9-bf20-4091b26fc01d admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:42.937488 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e25b10-9275-4ff9-bf20-4091b26fc01d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:42.939321 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e25b10-9275-4ff9-bf20-4091b26fc01d admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:42.955350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b0e86-206d-42e8-90d5-2cc141d05e03 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:42.956514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1358/2724] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 05:14:42.967060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b0e86-206d-42e8-90d5-2cc141d05e03 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:42.972921 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006a7d9-8d68-43fe-a2fd-2eb8b13b04c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:42.973836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006a7d9-8d68-43fe-a2fd-2eb8b13b04c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:42.974228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006a7d9-8d68-43fe-a2fd-2eb8b13b04c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:42.974613 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006a7d9-8d68-43fe-a2fd-2eb8b13b04c8 admin admin] PATH_INFO: `/v3/projects/bf53ada8ecf846428896035687708bd1/users/88f0a82e2169489083061f5822216700/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:42.994323 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006a7d9-8d68-43fe-a2fd-2eb8b13b04c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf53ada8ecf846428896035687708bd1, user_id=88f0a82e2169489083061f5822216700, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:42.997195 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006a7d9-8d68-43fe-a2fd-2eb8b13b04c8 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:43.017799 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1367/2725] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:14:42 2022] PUT /identity/v3/projects/bf53ada8ecf846428896035687708bd1/users/88f0a82e2169489083061f5822216700/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 05:14:43.029235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555845dd-9e34-497d-b309-50d1677ad773 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:43.029615 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555845dd-9e34-497d-b309-50d1677ad773 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:43.030020 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555845dd-9e34-497d-b309-50d1677ad773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:43.081046 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-555845dd-9e34-497d-b309-50d1677ad773 None None] MFA Rules not processed for user `88f0a82e2169489083061f5822216700`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:43.084597 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-555845dd-9e34-497d-b309-50d1677ad773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:43.212179 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1359/2726] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:43.223738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006a7d9-8d68-43fe-a2fd-2eb8b13b04c8 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:43.226406 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9006a7d9-8d68-43fe-a2fd-2eb8b13b04c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:43.291482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:43.349968 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006a7d9-8d68-43fe-a2fd-2eb8b13b04c8 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:43.357163 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e00f0-e1f8-4f6e-a285-25c5f4443c59 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b77a955866c54a43a7aaa271fefdec57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:43.357759 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e00f0-e1f8-4f6e-a285-25c5f4443c59 service glance] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:43.358256 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e00f0-e1f8-4f6e-a285-25c5f4443c59 service glance] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:43.358693 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e00f0-e1f8-4f6e-a285-25c5f4443c59 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:43.370395 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e00f0-e1f8-4f6e-a285-25c5f4443c59 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:43.372367 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e00f0-e1f8-4f6e-a285-25c5f4443c59 service glance] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:43.385181 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1368/2727] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:46.431565 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555845dd-9e34-497d-b309-50d1677ad773 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:46.432991 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-555845dd-9e34-497d-b309-50d1677ad773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:46.467645 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:46.508458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555845dd-9e34-497d-b309-50d1677ad773 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:46.513493 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc4848-c69b-4890-9501-c2396e84fa0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:46.514172 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc4848-c69b-4890-9501-c2396e84fa0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:46.514420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc4848-c69b-4890-9501-c2396e84fa0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:46.515107 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc4848-c69b-4890-9501-c2396e84fa0b admin admin] PATH_INFO: `/v3/users/fb67730cd96c4d329af1abf66199e0ca` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:46.515965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc4848-c69b-4890-9501-c2396e84fa0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb67730cd96c4d329af1abf66199e0ca)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:46.517958 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc4848-c69b-4890-9501-c2396e84fa0b admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:46.566409 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-25cc4848-c69b-4890-9501-c2396e84fa0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb67730cd96c4d329af1abf66199e0ca'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:46.567055 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-25cc4848-c69b-4890-9501-c2396e84fa0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb67730cd96c4d329af1abf66199e0ca'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:46.585429 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-25cc4848-c69b-4890-9501-c2396e84fa0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb67730cd96c4d329af1abf66199e0ca'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:46.593869 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-25cc4848-c69b-4890-9501-c2396e84fa0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb67730cd96c4d329af1abf66199e0ca'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:46.594757 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-25cc4848-c69b-4890-9501-c2396e84fa0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb67730cd96c4d329af1abf66199e0ca'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:46.608039 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1360/2728] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:14:46 2022] DELETE /identity/v3/users/fb67730cd96c4d329af1abf66199e0ca => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:46.612506 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81a4cb-c165-4807-b065-6ff815f2bdb8 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:46.612978 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81a4cb-c165-4807-b065-6ff815f2bdb8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:46.613123 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81a4cb-c165-4807-b065-6ff815f2bdb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:46.633674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-6a81a4cb-c165-4807-b065-6ff815f2bdb8 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:46.634875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6a81a4cb-c165-4807-b065-6ff815f2bdb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:46.700092 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1369/2729] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:46.710139 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc4848-c69b-4890-9501-c2396e84fa0b admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:46.711789 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-25cc4848-c69b-4890-9501-c2396e84fa0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:46.769565 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc4848-c69b-4890-9501-c2396e84fa0b admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:46.777588 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7aff96-44ae-43d4-9726-d429f766a2f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:46.777588 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7aff96-44ae-43d4-9726-d429f766a2f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:46.777910 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7aff96-44ae-43d4-9726-d429f766a2f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:46.778405 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7aff96-44ae-43d4-9726-d429f766a2f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:46.790505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7aff96-44ae-43d4-9726-d429f766a2f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:46.791919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7aff96-44ae-43d4-9726-d429f766a2f0 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:46.801277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1361/2730] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:47.089016 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81a4cb-c165-4807-b065-6ff815f2bdb8 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:47.090584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6a81a4cb-c165-4807-b065-6ff815f2bdb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:47.108656 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81a4cb-c165-4807-b065-6ff815f2bdb8 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:47.113200 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092c33a-2eb5-4240-a025-8d53683e78bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:47.113865 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092c33a-2eb5-4240-a025-8d53683e78bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:47.114141 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092c33a-2eb5-4240-a025-8d53683e78bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:47.114420 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092c33a-2eb5-4240-a025-8d53683e78bd admin admin] PATH_INFO: `/v3/projects/93c213e8c4fd4c51a625e6c2ccf8bcf7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:47.115555 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092c33a-2eb5-4240-a025-8d53683e78bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=93c213e8c4fd4c51a625e6c2ccf8bcf7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:47.118015 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092c33a-2eb5-4240-a025-8d53683e78bd admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:47.173310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c092c33a-2eb5-4240-a025-8d53683e78bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93c213e8c4fd4c51a625e6c2ccf8bcf7', 'request_id': 'req-c092c33a-2eb5-4240-a025-8d53683e78bd'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:47.182078 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c092c33a-2eb5-4240-a025-8d53683e78bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93c213e8c4fd4c51a625e6c2ccf8bcf7', 'request_id': 'req-c092c33a-2eb5-4240-a025-8d53683e78bd'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:47.188136 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c092c33a-2eb5-4240-a025-8d53683e78bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93c213e8c4fd4c51a625e6c2ccf8bcf7', 'request_id': 'req-c092c33a-2eb5-4240-a025-8d53683e78bd'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:47.207641 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c092c33a-2eb5-4240-a025-8d53683e78bd admin admin] The token cache is being invalidate because project 93c213e8c4fd4c51a625e6c2ccf8bcf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:14:47.208093 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c092c33a-2eb5-4240-a025-8d53683e78bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:47.208772 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-c092c33a-2eb5-4240-a025-8d53683e78bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:47.210713 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1370/2731] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:14:47 2022] DELETE /identity/v3/projects/93c213e8c4fd4c51a625e6c2ccf8bcf7 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:47.214360 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7aff96-44ae-43d4-9726-d429f766a2f0 service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:47.224341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7aff96-44ae-43d4-9726-d429f766a2f0 service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:47.276761 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:47.297676 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1fccd4-b39a-4aa4-a8b5-b7a9770e98f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:47.298330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fccd4-b39a-4aa4-a8b5-b7a9770e98f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:47.298611 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fccd4-b39a-4aa4-a8b5-b7a9770e98f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:47.298887 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fccd4-b39a-4aa4-a8b5-b7a9770e98f6 admin admin] PATH_INFO: `/v3/users/2ca0b72be13d4365b6ffbe71eea5932e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:47.300180 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1fccd4-b39a-4aa4-a8b5-b7a9770e98f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ca0b72be13d4365b6ffbe71eea5932e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:47.302174 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1fccd4-b39a-4aa4-a8b5-b7a9770e98f6 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:47.353195 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-1d1fccd4-b39a-4aa4-a8b5-b7a9770e98f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ca0b72be13d4365b6ffbe71eea5932e'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:47.353343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-1d1fccd4-b39a-4aa4-a8b5-b7a9770e98f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ca0b72be13d4365b6ffbe71eea5932e'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:47.366569 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-1d1fccd4-b39a-4aa4-a8b5-b7a9770e98f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ca0b72be13d4365b6ffbe71eea5932e'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:47.374974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-1d1fccd4-b39a-4aa4-a8b5-b7a9770e98f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ca0b72be13d4365b6ffbe71eea5932e'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:47.375612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-1d1fccd4-b39a-4aa4-a8b5-b7a9770e98f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ca0b72be13d4365b6ffbe71eea5932e'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:47.388990 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1362/2732] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:14:47 2022] DELETE /identity/v3/users/2ca0b72be13d4365b6ffbe71eea5932e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:47.949048 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092c33a-2eb5-4240-a025-8d53683e78bd admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:47.950520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c092c33a-2eb5-4240-a025-8d53683e78bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:47.964423 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092c33a-2eb5-4240-a025-8d53683e78bd admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:48.015311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:48.035419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a0dca-31cd-469d-b353-31d2c822472e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:48.036060 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a0dca-31cd-469d-b353-31d2c822472e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:48.036341 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a0dca-31cd-469d-b353-31d2c822472e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:48.036617 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a0dca-31cd-469d-b353-31d2c822472e admin admin] PATH_INFO: `/v3/projects/621fc54ce675483f8edb0b709095e6c2` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:48.038648 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a0dca-31cd-469d-b353-31d2c822472e admin admin] RBAC: Authorizing `identity:delete_project(project_id=621fc54ce675483f8edb0b709095e6c2)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:48.040663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a0dca-31cd-469d-b353-31d2c822472e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:48.092209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-185a0dca-31cd-469d-b353-31d2c822472e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '621fc54ce675483f8edb0b709095e6c2', 'request_id': 'req-185a0dca-31cd-469d-b353-31d2c822472e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:48.099194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-185a0dca-31cd-469d-b353-31d2c822472e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '621fc54ce675483f8edb0b709095e6c2', 'request_id': 'req-185a0dca-31cd-469d-b353-31d2c822472e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:48.104864 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-185a0dca-31cd-469d-b353-31d2c822472e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '621fc54ce675483f8edb0b709095e6c2', 'request_id': 'req-185a0dca-31cd-469d-b353-31d2c822472e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:48.115373 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-185a0dca-31cd-469d-b353-31d2c822472e admin admin] The token cache is being invalidate because project 621fc54ce675483f8edb0b709095e6c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:14:48.115659 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-185a0dca-31cd-469d-b353-31d2c822472e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:48.116287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-185a0dca-31cd-469d-b353-31d2c822472e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:48.117805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1371/2733] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:14:47 2022] DELETE /identity/v3/projects/621fc54ce675483f8edb0b709095e6c2 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:48.122005 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1fccd4-b39a-4aa4-a8b5-b7a9770e98f6 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:48.123608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-1d1fccd4-b39a-4aa4-a8b5-b7a9770e98f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:48.142293 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1fccd4-b39a-4aa4-a8b5-b7a9770e98f6 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:48.202950 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce13d0-f1a0-4702-a461-fd583940dcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:48.203590 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce13d0-f1a0-4702-a461-fd583940dcdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:48.203865 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce13d0-f1a0-4702-a461-fd583940dcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:48.204137 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce13d0-f1a0-4702-a461-fd583940dcdd admin admin] PATH_INFO: `/v3/users/b9a0c47eca6d4e73ab3f0d691e653d7b` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:48.205300 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce13d0-f1a0-4702-a461-fd583940dcdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9a0c47eca6d4e73ab3f0d691e653d7b)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:48.207217 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce13d0-f1a0-4702-a461-fd583940dcdd admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:48.258612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-85ce13d0-f1a0-4702-a461-fd583940dcdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9a0c47eca6d4e73ab3f0d691e653d7b'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:48.259437 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-85ce13d0-f1a0-4702-a461-fd583940dcdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9a0c47eca6d4e73ab3f0d691e653d7b'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:48.271787 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-85ce13d0-f1a0-4702-a461-fd583940dcdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9a0c47eca6d4e73ab3f0d691e653d7b'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:48.278619 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-85ce13d0-f1a0-4702-a461-fd583940dcdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9a0c47eca6d4e73ab3f0d691e653d7b'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:48.279338 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-85ce13d0-f1a0-4702-a461-fd583940dcdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9a0c47eca6d4e73ab3f0d691e653d7b'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:48.280937 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:48.302724 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1363/2734] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:14:48 2022] DELETE /identity/v3/users/b9a0c47eca6d4e73ab3f0d691e653d7b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:48.590737 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a0dca-31cd-469d-b353-31d2c822472e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:48.592476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-185a0dca-31cd-469d-b353-31d2c822472e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:48.606242 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a0dca-31cd-469d-b353-31d2c822472e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:48.665193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1338b0c-ee5a-4736-ad24-5b4c7f61487e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:48.665887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1338b0c-ee5a-4736-ad24-5b4c7f61487e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:48.666171 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1338b0c-ee5a-4736-ad24-5b4c7f61487e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:48.666451 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1338b0c-ee5a-4736-ad24-5b4c7f61487e admin admin] PATH_INFO: `/v3/projects/1aec933a744649b6b32d013e946f40a5` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:48.667627 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1338b0c-ee5a-4736-ad24-5b4c7f61487e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aec933a744649b6b32d013e946f40a5)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:48.669667 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1338b0c-ee5a-4736-ad24-5b4c7f61487e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:48.718861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a1338b0c-ee5a-4736-ad24-5b4c7f61487e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aec933a744649b6b32d013e946f40a5', 'request_id': 'req-a1338b0c-ee5a-4736-ad24-5b4c7f61487e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:48.727201 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a1338b0c-ee5a-4736-ad24-5b4c7f61487e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aec933a744649b6b32d013e946f40a5', 'request_id': 'req-a1338b0c-ee5a-4736-ad24-5b4c7f61487e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:48.736975 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a1338b0c-ee5a-4736-ad24-5b4c7f61487e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aec933a744649b6b32d013e946f40a5', 'request_id': 'req-a1338b0c-ee5a-4736-ad24-5b4c7f61487e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:48.746307 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:48.750470 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a1338b0c-ee5a-4736-ad24-5b4c7f61487e admin admin] The token cache is being invalidate because project 1aec933a744649b6b32d013e946f40a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:14:48.750777 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a1338b0c-ee5a-4736-ad24-5b4c7f61487e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:48.751844 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a1338b0c-ee5a-4736-ad24-5b4c7f61487e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:14:48.756636 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1372/2735] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:14:48 2022] DELETE /identity/v3/projects/1aec933a744649b6b32d013e946f40a5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:48.759097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d19e19-b5fd-4657-b990-3d9324e832d9 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:48.759365 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d19e19-b5fd-4657-b990-3d9324e832d9 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:48.759722 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d19e19-b5fd-4657-b990-3d9324e832d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:48.795050 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-67d19e19-b5fd-4657-b990-3d9324e832d9 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:48.796285 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-67d19e19-b5fd-4657-b990-3d9324e832d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:48.881634 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1364/2736] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:48.886656 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ad918-5ca0-45d6-bd42-e0c9ffa8e697 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:48.886851 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ad918-5ca0-45d6-bd42-e0c9ffa8e697 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:48.887027 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ad918-5ca0-45d6-bd42-e0c9ffa8e697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:48.896272 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884be1e4-738c-4c58-b48c-5c6f13b2d38c None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:48.896631 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884be1e4-738c-4c58-b48c-5c6f13b2d38c None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:48.897015 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884be1e4-738c-4c58-b48c-5c6f13b2d38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:48.917728 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-6a9ad918-5ca0-45d6-bd42-e0c9ffa8e697 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:48.918993 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6a9ad918-5ca0-45d6-bd42-e0c9ffa8e697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:48.922316 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-884be1e4-738c-4c58-b48c-5c6f13b2d38c None None] MFA Rules not processed for user `30c0ff27c5a84ebd8615fb4608fc3882`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:48.923536 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-884be1e4-738c-4c58-b48c-5c6f13b2d38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:48.933333 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1373/2737] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:14:48.937389 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ad918-5ca0-45d6-bd42-e0c9ffa8e697 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:48.947499 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ad918-5ca0-45d6-bd42-e0c9ffa8e697 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:48.952433 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8abeb-10f4-4f8e-a105-c6abccc29d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:48.953035 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8abeb-10f4-4f8e-a105-c6abccc29d79 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:48.953626 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8abeb-10f4-4f8e-a105-c6abccc29d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:48.953626 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8abeb-10f4-4f8e-a105-c6abccc29d79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:48.954326 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8abeb-10f4-4f8e-a105-c6abccc29d79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 05:14:48.954630 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8abeb-10f4-4f8e-a105-c6abccc29d79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:48.956215 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8abeb-10f4-4f8e-a105-c6abccc29d79 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:48.976270 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1374/2738] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 05:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 05:14:48.977752 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8abeb-10f4-4f8e-a105-c6abccc29d79 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:48.987858 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8abeb-10f4-4f8e-a105-c6abccc29d79 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:48.988349 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1365/2739] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 05:14:48.993253 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f1cad-840b-41eb-9874-a0aa22b2f9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:48.994014 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f1cad-840b-41eb-9874-a0aa22b2f9f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:48.994372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f1cad-840b-41eb-9874-a0aa22b2f9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:48.994652 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f1cad-840b-41eb-9874-a0aa22b2f9f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:48.995322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f1cad-840b-41eb-9874-a0aa22b2f9f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:48.997213 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f1cad-840b-41eb-9874-a0aa22b2f9f1 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:49.034992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1375/2740] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:14:48 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:14:49.035184 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884be1e4-738c-4c58-b48c-5c6f13b2d38c None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:49.049936 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884be1e4-738c-4c58-b48c-5c6f13b2d38c None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:49.134133 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617f50a-ecdb-4bf2-8331-0c5932c5e810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:49.134915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617f50a-ecdb-4bf2-8331-0c5932c5e810 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:49.135324 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617f50a-ecdb-4bf2-8331-0c5932c5e810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:49.135704 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617f50a-ecdb-4bf2-8331-0c5932c5e810 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:49.136512 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617f50a-ecdb-4bf2-8331-0c5932c5e810 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:49.138551 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617f50a-ecdb-4bf2-8331-0c5932c5e810 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:49.164996 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1366/2741] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:14:49 2022] POST /identity/v3/users => generated 288 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:14:49.169408 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f1cad-840b-41eb-9874-a0aa22b2f9f1 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:49.179751 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f1cad-840b-41eb-9874-a0aa22b2f9f1 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:49.185591 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0168a7d-8dea-46ea-a24a-0a4280c2e26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:49.186177 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0168a7d-8dea-46ea-a24a-0a4280c2e26e admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:49.186370 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0168a7d-8dea-46ea-a24a-0a4280c2e26e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:49.186648 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0168a7d-8dea-46ea-a24a-0a4280c2e26e admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:49.187342 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0168a7d-8dea-46ea-a24a-0a4280c2e26e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:49.188924 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0168a7d-8dea-46ea-a24a-0a4280c2e26e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:49.206019 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1376/2742] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 05:14:49.207647 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617f50a-ecdb-4bf2-8331-0c5932c5e810 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:49.218390 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617f50a-ecdb-4bf2-8331-0c5932c5e810 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:49.223217 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6eccd8-b146-4460-b1e8-99519ff0a1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:49.223835 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6eccd8-b146-4460-b1e8-99519ff0a1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:49.224114 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6eccd8-b146-4460-b1e8-99519ff0a1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:49.224389 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6eccd8-b146-4460-b1e8-99519ff0a1e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:49.225127 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6eccd8-b146-4460-b1e8-99519ff0a1e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:49.227784 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6eccd8-b146-4460-b1e8-99519ff0a1e8 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:49.240861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1367/2743] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 05:14:49.245923 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0168a7d-8dea-46ea-a24a-0a4280c2e26e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:49.256475 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6eccd8-b146-4460-b1e8-99519ff0a1e8 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:49.259319 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5a6eccd8-b146-4460-b1e8-99519ff0a1e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:49.260172 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0168a7d-8dea-46ea-a24a-0a4280c2e26e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:49.273382 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0c92a-bcbb-4435-bfe2-0dc82fbdb5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:49.274867 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0c92a-bcbb-4435-bfe2-0dc82fbdb5ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:49.275384 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0c92a-bcbb-4435-bfe2-0dc82fbdb5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:49.276315 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0c92a-bcbb-4435-bfe2-0dc82fbdb5ad admin admin] PATH_INFO: `/v3/projects/6d9d43ab54154cf6a87ad71eee803065/users/768c9028339f4b03b45dbdda351469d7/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:49.283181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:14:49.297462 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a0c92a-bcbb-4435-bfe2-0dc82fbdb5ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d9d43ab54154cf6a87ad71eee803065, user_id=768c9028339f4b03b45dbdda351469d7, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:49.306153 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a0c92a-bcbb-4435-bfe2-0dc82fbdb5ad admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:49.327157 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1377/2744] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:14:49 2022] PUT /identity/v3/projects/6d9d43ab54154cf6a87ad71eee803065/users/768c9028339f4b03b45dbdda351469d7/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:14:49.337356 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89294417-c8f0-45e9-8566-89421202c685 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:49.338006 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89294417-c8f0-45e9-8566-89421202c685 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:49.338912 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89294417-c8f0-45e9-8566-89421202c685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:49.387302 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-89294417-c8f0-45e9-8566-89421202c685 None None] MFA Rules not processed for user `768c9028339f4b03b45dbdda351469d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:14:49.388770 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-89294417-c8f0-45e9-8566-89421202c685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:49.405445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6eccd8-b146-4460-b1e8-99519ff0a1e8 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:49.472871 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1378/2745] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:14:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 05:14:49.473802 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89294417-c8f0-45e9-8566-89421202c685 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:49.475146 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-89294417-c8f0-45e9-8566-89421202c685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:49.480918 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a5077-79d8-4a46-afab-2d361d9000e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:49.481729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a5077-79d8-4a46-afab-2d361d9000e4 service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:49.482539 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a5077-79d8-4a46-afab-2d361d9000e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:49.482884 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a5077-79d8-4a46-afab-2d361d9000e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:49.484758 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-8b7a5077-79d8-4a46-afab-2d361d9000e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:49.498680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a5077-79d8-4a46-afab-2d361d9000e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:49.501641 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a5077-79d8-4a46-afab-2d361d9000e4 service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:49.512854 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1368/2746] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:49.513645 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a5077-79d8-4a46-afab-2d361d9000e4 service nova] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:49.514971 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-8b7a5077-79d8-4a46-afab-2d361d9000e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:49.547656 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89294417-c8f0-45e9-8566-89421202c685 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:49.551645 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea471654-2544-483f-89b7-d119cf97d1b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:49.552479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea471654-2544-483f-89b7-d119cf97d1b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:49.552868 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea471654-2544-483f-89b7-d119cf97d1b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:49.553144 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea471654-2544-483f-89b7-d119cf97d1b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:49.557663 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ea471654-2544-483f-89b7-d119cf97d1b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:49.569708 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea471654-2544-483f-89b7-d119cf97d1b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:49.571369 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea471654-2544-483f-89b7-d119cf97d1b1 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:49.583961 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1379/2747] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:49.590082 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a5077-79d8-4a46-afab-2d361d9000e4 service nova] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:49.594828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cb326-1aa1-474e-a138-ae1c374ec49f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:49.595630 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cb326-1aa1-474e-a138-ae1c374ec49f service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:49.596083 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cb326-1aa1-474e-a138-ae1c374ec49f service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:49.596433 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cb326-1aa1-474e-a138-ae1c374ec49f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:49.611796 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72cb326-1aa1-474e-a138-ae1c374ec49f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:49.613398 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72cb326-1aa1-474e-a138-ae1c374ec49f service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:49.625602 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1369/2748] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:53.395844 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea471654-2544-483f-89b7-d119cf97d1b1 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:14:53.415707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea471654-2544-483f-89b7-d119cf97d1b1 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:14:53.427550 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713b142-2d6e-4c9d-a633-05da2dc4846c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:14:53.429432 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713b142-2d6e-4c9d-a633-05da2dc4846c service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:14:53.430720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713b142-2d6e-4c9d-a633-05da2dc4846c service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:14:53.430720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713b142-2d6e-4c9d-a633-05da2dc4846c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:14:53.433499 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6713b142-2d6e-4c9d-a633-05da2dc4846c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:14:53.472758 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713b142-2d6e-4c9d-a633-05da2dc4846c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:14:53.478140 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713b142-2d6e-4c9d-a633-05da2dc4846c service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:14:53.501669 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1380/2749] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:14:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:14:59.621220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=82 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:15:00.229715 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cb326-1aa1-474e-a138-ae1c374ec49f service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:00.297519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cb326-1aa1-474e-a138-ae1c374ec49f service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:00.302740 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cfac25-7631-4c6a-af63-80adcec956ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:00.303662 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfac25-7631-4c6a-af63-80adcec956ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:00.303945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfac25-7631-4c6a-af63-80adcec956ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:00.304525 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfac25-7631-4c6a-af63-80adcec956ea admin admin] PATH_INFO: `/v3/users/768c9028339f4b03b45dbdda351469d7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:00.305816 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cfac25-7631-4c6a-af63-80adcec956ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=768c9028339f4b03b45dbdda351469d7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:00.307713 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cfac25-7631-4c6a-af63-80adcec956ea admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:00.358992 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-15cfac25-7631-4c6a-af63-80adcec956ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '768c9028339f4b03b45dbdda351469d7'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:00.359709 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-15cfac25-7631-4c6a-af63-80adcec956ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '768c9028339f4b03b45dbdda351469d7'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:00.372520 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-15cfac25-7631-4c6a-af63-80adcec956ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '768c9028339f4b03b45dbdda351469d7'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:00.382264 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-15cfac25-7631-4c6a-af63-80adcec956ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '768c9028339f4b03b45dbdda351469d7'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:00.383169 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-15cfac25-7631-4c6a-af63-80adcec956ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '768c9028339f4b03b45dbdda351469d7'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:00.402189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1370/2750] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:15:00 2022] DELETE /identity/v3/users/768c9028339f4b03b45dbdda351469d7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:15:00.407184 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868f5ac-896e-4ba2-b5b6-e617f5f8517b None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:00.407709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868f5ac-896e-4ba2-b5b6-e617f5f8517b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:00.407709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868f5ac-896e-4ba2-b5b6-e617f5f8517b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:00.415950 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:15:00.435944 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b868f5ac-896e-4ba2-b5b6-e617f5f8517b None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:15:00.437209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b868f5ac-896e-4ba2-b5b6-e617f5f8517b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:00.513477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1381/2751] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:15:00.526454 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cfac25-7631-4c6a-af63-80adcec956ea admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:00.528125 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-15cfac25-7631-4c6a-af63-80adcec956ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:00.601075 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cfac25-7631-4c6a-af63-80adcec956ea admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:00.604949 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29497e6b-7a83-4c7e-9ef9-6987ea9f9ec5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:00.605612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29497e6b-7a83-4c7e-9ef9-6987ea9f9ec5 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:00.605919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29497e6b-7a83-4c7e-9ef9-6987ea9f9ec5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:00.606192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29497e6b-7a83-4c7e-9ef9-6987ea9f9ec5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:00.617804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29497e6b-7a83-4c7e-9ef9-6987ea9f9ec5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:00.619292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29497e6b-7a83-4c7e-9ef9-6987ea9f9ec5 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:00.634480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1371/2752] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:15:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:15:00.917612 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868f5ac-896e-4ba2-b5b6-e617f5f8517b None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:00.919420 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b868f5ac-896e-4ba2-b5b6-e617f5f8517b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:00.932939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868f5ac-896e-4ba2-b5b6-e617f5f8517b None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:00.937909 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e1693-920d-4e68-93c4-2ad787af3894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:00.938566 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e1693-920d-4e68-93c4-2ad787af3894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:00.938841 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e1693-920d-4e68-93c4-2ad787af3894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:00.939123 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e1693-920d-4e68-93c4-2ad787af3894 admin admin] PATH_INFO: `/v3/projects/6d9d43ab54154cf6a87ad71eee803065` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:00.940308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e1693-920d-4e68-93c4-2ad787af3894 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d9d43ab54154cf6a87ad71eee803065)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:00.942352 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e1693-920d-4e68-93c4-2ad787af3894 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:00.990511 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-740e1693-920d-4e68-93c4-2ad787af3894 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d9d43ab54154cf6a87ad71eee803065', 'request_id': 'req-740e1693-920d-4e68-93c4-2ad787af3894'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:00.998485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-740e1693-920d-4e68-93c4-2ad787af3894 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d9d43ab54154cf6a87ad71eee803065', 'request_id': 'req-740e1693-920d-4e68-93c4-2ad787af3894'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:01.004204 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-740e1693-920d-4e68-93c4-2ad787af3894 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d9d43ab54154cf6a87ad71eee803065', 'request_id': 'req-740e1693-920d-4e68-93c4-2ad787af3894'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:01.015875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-740e1693-920d-4e68-93c4-2ad787af3894 admin admin] The token cache is being invalidate because project 6d9d43ab54154cf6a87ad71eee803065 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:15:01.016159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-740e1693-920d-4e68-93c4-2ad787af3894 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:01.016798 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-740e1693-920d-4e68-93c4-2ad787af3894 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:01.018398 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1382/2753] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:15:00 2022] DELETE /identity/v3/projects/6d9d43ab54154cf6a87ad71eee803065 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:15:01.024074 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256a6ae-5c4b-4123-99c4-660c73515eed None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:01.024390 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256a6ae-5c4b-4123-99c4-660c73515eed None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:01.024741 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256a6ae-5c4b-4123-99c4-660c73515eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:01.045079 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-9256a6ae-5c4b-4123-99c4-660c73515eed None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:15:01.046353 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-9256a6ae-5c4b-4123-99c4-660c73515eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:01.063417 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:15:01.110679 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1372/2754] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:15:01.119819 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a146e1-10d5-4f73-8464-c07bc1bdacfc None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:01.120035 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a146e1-10d5-4f73-8464-c07bc1bdacfc None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:01.120311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a146e1-10d5-4f73-8464-c07bc1bdacfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:01.143949 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-14a146e1-10d5-4f73-8464-c07bc1bdacfc None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:15:01.144699 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-14a146e1-10d5-4f73-8464-c07bc1bdacfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:01.156116 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1383/2755] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:15:01.161697 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256a6ae-5c4b-4123-99c4-660c73515eed None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:01.172832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256a6ae-5c4b-4123-99c4-660c73515eed None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:01.178033 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff95fb6-e08f-4ec3-8036-c46efe070410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:01.179235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff95fb6-e08f-4ec3-8036-c46efe070410 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:01.179519 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff95fb6-e08f-4ec3-8036-c46efe070410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:01.179790 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff95fb6-e08f-4ec3-8036-c46efe070410 admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:01.180496 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff95fb6-e08f-4ec3-8036-c46efe070410 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 05:15:01.180805 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff95fb6-e08f-4ec3-8036-c46efe070410 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:01.182847 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff95fb6-e08f-4ec3-8036-c46efe070410 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:01.196671 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1373/2756] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 05:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 05:15:01.200486 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a146e1-10d5-4f73-8464-c07bc1bdacfc None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:01.210637 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a146e1-10d5-4f73-8464-c07bc1bdacfc None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:01.218171 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761d275-5671-47cb-8bcb-b8d66dbb7ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:01.219333 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761d275-5671-47cb-8bcb-b8d66dbb7ee6 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:01.219718 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761d275-5671-47cb-8bcb-b8d66dbb7ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:01.220142 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761d275-5671-47cb-8bcb-b8d66dbb7ee6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:01.220954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0761d275-5671-47cb-8bcb-b8d66dbb7ee6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:01.223563 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0761d275-5671-47cb-8bcb-b8d66dbb7ee6 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:01.258054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1384/2757] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:15:01 2022] POST /identity/v3/projects => generated 391 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 05:15:01.262288 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff95fb6-e08f-4ec3-8036-c46efe070410 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:01.273243 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff95fb6-e08f-4ec3-8036-c46efe070410 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:01.337353 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd7aa54-3b0c-43e9-ac88-e772176c7d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:01.337949 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7aa54-3b0c-43e9-ac88-e772176c7d1a admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:01.338232 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7aa54-3b0c-43e9-ac88-e772176c7d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:01.338509 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7aa54-3b0c-43e9-ac88-e772176c7d1a admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:01.339194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd7aa54-3b0c-43e9-ac88-e772176c7d1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:01.341089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd7aa54-3b0c-43e9-ac88-e772176c7d1a admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:01.363882 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1374/2758] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:15:01 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 05:15:01.368880 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761d275-5671-47cb-8bcb-b8d66dbb7ee6 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:01.379021 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761d275-5671-47cb-8bcb-b8d66dbb7ee6 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:01.383701 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f9f76-15b3-4876-9bde-0bbf9de4d2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:01.384310 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f9f76-15b3-4876-9bde-0bbf9de4d2dd admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:01.384584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f9f76-15b3-4876-9bde-0bbf9de4d2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:01.384853 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f9f76-15b3-4876-9bde-0bbf9de4d2dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:01.385649 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f9f76-15b3-4876-9bde-0bbf9de4d2dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:01.387752 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f9f76-15b3-4876-9bde-0bbf9de4d2dd admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:01.398509 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1385/2759] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:15:01.402719 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd7aa54-3b0c-43e9-ac88-e772176c7d1a admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:01.413110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd7aa54-3b0c-43e9-ac88-e772176c7d1a admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:01.417908 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9376e5b-0f0c-435a-9ff3-7fe4f217e510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:01.418509 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9376e5b-0f0c-435a-9ff3-7fe4f217e510 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:01.418786 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9376e5b-0f0c-435a-9ff3-7fe4f217e510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:01.419060 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9376e5b-0f0c-435a-9ff3-7fe4f217e510 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:01.419746 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9376e5b-0f0c-435a-9ff3-7fe4f217e510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:01.421307 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9376e5b-0f0c-435a-9ff3-7fe4f217e510 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:01.434156 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1375/2760] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:15:01.438593 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f9f76-15b3-4876-9bde-0bbf9de4d2dd admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:01.449209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f9f76-15b3-4876-9bde-0bbf9de4d2dd admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:01.453905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00440ab0-ad65-410d-bfda-6bf47a772583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:01.454568 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00440ab0-ad65-410d-bfda-6bf47a772583 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:01.454854 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00440ab0-ad65-410d-bfda-6bf47a772583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:01.455139 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00440ab0-ad65-410d-bfda-6bf47a772583 admin admin] PATH_INFO: `/v3/projects/4589bd90068b4b2cbeb7e655869286cd/users/ea7da377497343f0a2d11c9d2260d7dc/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:01.473667 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00440ab0-ad65-410d-bfda-6bf47a772583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4589bd90068b4b2cbeb7e655869286cd, user_id=ea7da377497343f0a2d11c9d2260d7dc, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:01.476849 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00440ab0-ad65-410d-bfda-6bf47a772583 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:01.489808 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1386/2761] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:15:01 2022] PUT /identity/v3/projects/4589bd90068b4b2cbeb7e655869286cd/users/ea7da377497343f0a2d11c9d2260d7dc/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:15:01.500895 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471f0a9-59ed-4edd-b0b4-3a850555bcd3 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:01.501183 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471f0a9-59ed-4edd-b0b4-3a850555bcd3 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:01.501460 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471f0a9-59ed-4edd-b0b4-3a850555bcd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:01.531423 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-b471f0a9-59ed-4edd-b0b4-3a850555bcd3 None None] MFA Rules not processed for user `ea7da377497343f0a2d11c9d2260d7dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:15:01.532631 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-b471f0a9-59ed-4edd-b0b4-3a850555bcd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:01.599459 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1376/2762] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:15:01 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:15:01.611771 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00440ab0-ad65-410d-bfda-6bf47a772583 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:01.613605 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-00440ab0-ad65-410d-bfda-6bf47a772583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:01.640031 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:15:01.675350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00440ab0-ad65-410d-bfda-6bf47a772583 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:01.679632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928a46f-01cc-4bce-811d-e654475f20e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:01.680264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928a46f-01cc-4bce-811d-e654475f20e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:01.680528 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928a46f-01cc-4bce-811d-e654475f20e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:01.680800 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928a46f-01cc-4bce-811d-e654475f20e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:01.689133 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a928a46f-01cc-4bce-811d-e654475f20e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:01.690608 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a928a46f-01cc-4bce-811d-e654475f20e7 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:01.699736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1387/2763] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:15:01 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:15:11.596705 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:15:11.695818 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=16 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:15:14.483426 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471f0a9-59ed-4edd-b0b4-3a850555bcd3 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:14.484922 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-b471f0a9-59ed-4edd-b0b4-3a850555bcd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:14.556705 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471f0a9-59ed-4edd-b0b4-3a850555bcd3 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:14.560541 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8407fab-21ce-4b7d-98e7-175d6f2f986e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:14.561284 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8407fab-21ce-4b7d-98e7-175d6f2f986e service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:14.561595 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8407fab-21ce-4b7d-98e7-175d6f2f986e service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:14.561871 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8407fab-21ce-4b7d-98e7-175d6f2f986e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:14.563707 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-a8407fab-21ce-4b7d-98e7-175d6f2f986e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:14.643690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8407fab-21ce-4b7d-98e7-175d6f2f986e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:14.645198 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8407fab-21ce-4b7d-98e7-175d6f2f986e service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:14.660450 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1377/2764] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:15:14 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:15:19.083527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928a46f-01cc-4bce-811d-e654475f20e7 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:19.085087 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a928a46f-01cc-4bce-811d-e654475f20e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:19.154871 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928a46f-01cc-4bce-811d-e654475f20e7 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:19.159633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655d3f0-f836-4707-aea2-ea7e2d3f6c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:19.160278 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655d3f0-f836-4707-aea2-ea7e2d3f6c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:19.160564 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655d3f0-f836-4707-aea2-ea7e2d3f6c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:19.160835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655d3f0-f836-4707-aea2-ea7e2d3f6c68 admin admin] PATH_INFO: `/v3/users/88f0a82e2169489083061f5822216700` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:19.162661 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655d3f0-f836-4707-aea2-ea7e2d3f6c68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88f0a82e2169489083061f5822216700)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:19.164692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655d3f0-f836-4707-aea2-ea7e2d3f6c68 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:19.213122 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2655d3f0-f836-4707-aea2-ea7e2d3f6c68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88f0a82e2169489083061f5822216700'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:19.220211 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2655d3f0-f836-4707-aea2-ea7e2d3f6c68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88f0a82e2169489083061f5822216700'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:19.220919 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2655d3f0-f836-4707-aea2-ea7e2d3f6c68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88f0a82e2169489083061f5822216700'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:19.242185 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2655d3f0-f836-4707-aea2-ea7e2d3f6c68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88f0a82e2169489083061f5822216700'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:19.243159 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2655d3f0-f836-4707-aea2-ea7e2d3f6c68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88f0a82e2169489083061f5822216700'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:19.259544 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1388/2765] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:15:19 2022] DELETE /identity/v3/users/88f0a82e2169489083061f5822216700 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:15:19.264119 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f1b2d-da8b-4b3e-8880-f33c92915191 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:19.264894 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f1b2d-da8b-4b3e-8880-f33c92915191 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:19.265110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f1b2d-da8b-4b3e-8880-f33c92915191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:19.290465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-389f1b2d-da8b-4b3e-8880-f33c92915191 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:15:19.291685 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-389f1b2d-da8b-4b3e-8880-f33c92915191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:19.355931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1378/2766] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:15:19.367565 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655d3f0-f836-4707-aea2-ea7e2d3f6c68 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:19.369278 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2655d3f0-f836-4707-aea2-ea7e2d3f6c68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:19.450193 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655d3f0-f836-4707-aea2-ea7e2d3f6c68 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:19.453831 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfa9eb-f4e4-4a05-95eb-ebc7646f6d6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:19.454448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfa9eb-f4e4-4a05-95eb-ebc7646f6d6e service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:19.454736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfa9eb-f4e4-4a05-95eb-ebc7646f6d6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:19.455001 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfa9eb-f4e4-4a05-95eb-ebc7646f6d6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:19.464714 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cfa9eb-f4e4-4a05-95eb-ebc7646f6d6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:19.466181 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cfa9eb-f4e4-4a05-95eb-ebc7646f6d6e service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:19.476765 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1389/2767] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:15:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:15:19.775848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f1b2d-da8b-4b3e-8880-f33c92915191 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:19.776745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-389f1b2d-da8b-4b3e-8880-f33c92915191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:19.796669 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f1b2d-da8b-4b3e-8880-f33c92915191 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:19.804653 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735a0f3-c57e-443a-bcd2-528276e91098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:19.805327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735a0f3-c57e-443a-bcd2-528276e91098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:19.805678 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735a0f3-c57e-443a-bcd2-528276e91098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:19.806045 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735a0f3-c57e-443a-bcd2-528276e91098 admin admin] PATH_INFO: `/v3/projects/bf53ada8ecf846428896035687708bd1` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:19.807297 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735a0f3-c57e-443a-bcd2-528276e91098 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf53ada8ecf846428896035687708bd1)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:19.809297 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735a0f3-c57e-443a-bcd2-528276e91098 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:19.866862 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-3735a0f3-c57e-443a-bcd2-528276e91098 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf53ada8ecf846428896035687708bd1', 'request_id': 'req-3735a0f3-c57e-443a-bcd2-528276e91098'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:19.869886 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:15:19.878119 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-3735a0f3-c57e-443a-bcd2-528276e91098 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf53ada8ecf846428896035687708bd1', 'request_id': 'req-3735a0f3-c57e-443a-bcd2-528276e91098'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:19.886197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-3735a0f3-c57e-443a-bcd2-528276e91098 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf53ada8ecf846428896035687708bd1', 'request_id': 'req-3735a0f3-c57e-443a-bcd2-528276e91098'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:19.900552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-3735a0f3-c57e-443a-bcd2-528276e91098 admin admin] The token cache is being invalidate because project bf53ada8ecf846428896035687708bd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:15:19.900933 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-3735a0f3-c57e-443a-bcd2-528276e91098 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:19.901737 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-3735a0f3-c57e-443a-bcd2-528276e91098 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:19.903562 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1379/2768] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:15:19 2022] DELETE /identity/v3/projects/bf53ada8ecf846428896035687708bd1 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:15:23.274636 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfa9eb-f4e4-4a05-95eb-ebc7646f6d6e service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:23.276194 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-91cfa9eb-f4e4-4a05-95eb-ebc7646f6d6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:23.289427 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfa9eb-f4e4-4a05-95eb-ebc7646f6d6e service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:23.297815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:15:23.356601 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adcda8f-a703-46d5-9287-735d449d7b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:23.357255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcda8f-a703-46d5-9287-735d449d7b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:23.357532 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcda8f-a703-46d5-9287-735d449d7b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:23.357838 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcda8f-a703-46d5-9287-735d449d7b73 admin admin] PATH_INFO: `/v3/users/ea7da377497343f0a2d11c9d2260d7dc` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:23.359081 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adcda8f-a703-46d5-9287-735d449d7b73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea7da377497343f0a2d11c9d2260d7dc)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:23.361062 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adcda8f-a703-46d5-9287-735d449d7b73 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:23.422101 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0adcda8f-a703-46d5-9287-735d449d7b73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea7da377497343f0a2d11c9d2260d7dc'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:23.430992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0adcda8f-a703-46d5-9287-735d449d7b73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea7da377497343f0a2d11c9d2260d7dc'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:23.431839 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0adcda8f-a703-46d5-9287-735d449d7b73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea7da377497343f0a2d11c9d2260d7dc'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:23.446430 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0adcda8f-a703-46d5-9287-735d449d7b73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea7da377497343f0a2d11c9d2260d7dc'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:23.447182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0adcda8f-a703-46d5-9287-735d449d7b73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea7da377497343f0a2d11c9d2260d7dc'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:23.463122 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1390/2769] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:15:23 2022] DELETE /identity/v3/users/ea7da377497343f0a2d11c9d2260d7dc => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:15:23.467725 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49402e-53d2-4b71-93df-b5c9cf3d43e4 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:23.468267 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49402e-53d2-4b71-93df-b5c9cf3d43e4 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:23.468517 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49402e-53d2-4b71-93df-b5c9cf3d43e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:23.488953 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-5d49402e-53d2-4b71-93df-b5c9cf3d43e4 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:15:23.493532 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5d49402e-53d2-4b71-93df-b5c9cf3d43e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:23.564785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1380/2770] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:15:23.575100 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adcda8f-a703-46d5-9287-735d449d7b73 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:23.577021 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0adcda8f-a703-46d5-9287-735d449d7b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:23.775240 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adcda8f-a703-46d5-9287-735d449d7b73 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:23.778911 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff001a76-8371-4e2c-b9b5-f8e2c807e8c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:23.779532 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff001a76-8371-4e2c-b9b5-f8e2c807e8c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:23.779804 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff001a76-8371-4e2c-b9b5-f8e2c807e8c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:23.780071 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff001a76-8371-4e2c-b9b5-f8e2c807e8c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:23.791311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff001a76-8371-4e2c-b9b5-f8e2c807e8c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:23.792747 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff001a76-8371-4e2c-b9b5-f8e2c807e8c2 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:23.804583 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1391/2771] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:15:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:15:24.162966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49402e-53d2-4b71-93df-b5c9cf3d43e4 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:24.164633 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5d49402e-53d2-4b71-93df-b5c9cf3d43e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:24.179044 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49402e-53d2-4b71-93df-b5c9cf3d43e4 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:24.185812 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0104f9dc-870d-4548-bcf0-3cfc96b3fbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:24.186830 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104f9dc-870d-4548-bcf0-3cfc96b3fbce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:24.187224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104f9dc-870d-4548-bcf0-3cfc96b3fbce admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:24.187600 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104f9dc-870d-4548-bcf0-3cfc96b3fbce admin admin] PATH_INFO: `/v3/projects/4589bd90068b4b2cbeb7e655869286cd` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:24.189214 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0104f9dc-870d-4548-bcf0-3cfc96b3fbce admin admin] RBAC: Authorizing `identity:delete_project(project_id=4589bd90068b4b2cbeb7e655869286cd)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:24.191897 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0104f9dc-870d-4548-bcf0-3cfc96b3fbce admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:24.249603 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-0104f9dc-870d-4548-bcf0-3cfc96b3fbce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4589bd90068b4b2cbeb7e655869286cd', 'request_id': 'req-0104f9dc-870d-4548-bcf0-3cfc96b3fbce'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:24.261172 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-0104f9dc-870d-4548-bcf0-3cfc96b3fbce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4589bd90068b4b2cbeb7e655869286cd', 'request_id': 'req-0104f9dc-870d-4548-bcf0-3cfc96b3fbce'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:24.271482 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-0104f9dc-870d-4548-bcf0-3cfc96b3fbce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4589bd90068b4b2cbeb7e655869286cd', 'request_id': 'req-0104f9dc-870d-4548-bcf0-3cfc96b3fbce'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:24.289206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-0104f9dc-870d-4548-bcf0-3cfc96b3fbce admin admin] The token cache is being invalidate because project 4589bd90068b4b2cbeb7e655869286cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:15:24.289494 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-0104f9dc-870d-4548-bcf0-3cfc96b3fbce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:24.290193 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-0104f9dc-870d-4548-bcf0-3cfc96b3fbce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:15:24.291761 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1381/2772] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:15:24 2022] DELETE /identity/v3/projects/4589bd90068b4b2cbeb7e655869286cd => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:15:24.299771 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de9956-2e5b-4c13-b8c3-2ce6f9a27a2e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:24.300413 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de9956-2e5b-4c13-b8c3-2ce6f9a27a2e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:24.300679 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de9956-2e5b-4c13-b8c3-2ce6f9a27a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:24.332183 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-02de9956-2e5b-4c13-b8c3-2ce6f9a27a2e None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:15:24.335575 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-02de9956-2e5b-4c13-b8c3-2ce6f9a27a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:24.373477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:15:24.416318 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1392/2773] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:15:24.424827 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd44aee-104c-4729-8444-ff76563ad9e3 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:24.425109 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd44aee-104c-4729-8444-ff76563ad9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:24.425387 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd44aee-104c-4729-8444-ff76563ad9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:24.451476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-8fd44aee-104c-4729-8444-ff76563ad9e3 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:15:24.452830 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-8fd44aee-104c-4729-8444-ff76563ad9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:24.462099 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1382/2774] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:15:24.470626 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de9956-2e5b-4c13-b8c3-2ce6f9a27a2e None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:24.486877 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de9956-2e5b-4c13-b8c3-2ce6f9a27a2e None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:24.492275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8ed85-28b8-46c3-bb34-422d91f77130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:24.492909 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8ed85-28b8-46c3-bb34-422d91f77130 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:24.493192 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8ed85-28b8-46c3-bb34-422d91f77130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:24.493464 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8ed85-28b8-46c3-bb34-422d91f77130 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:24.494215 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e8ed85-28b8-46c3-bb34-422d91f77130 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 05:15:24.494518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e8ed85-28b8-46c3-bb34-422d91f77130 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:24.496128 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e8ed85-28b8-46c3-bb34-422d91f77130 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:24.510774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1393/2775] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 05:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 05:15:24.515889 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd44aee-104c-4729-8444-ff76563ad9e3 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:24.527006 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd44aee-104c-4729-8444-ff76563ad9e3 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:24.532197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3b2e3-e069-4c38-81fe-d8bb5098e5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:24.532832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3b2e3-e069-4c38-81fe-d8bb5098e5df admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:24.533109 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3b2e3-e069-4c38-81fe-d8bb5098e5df admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:24.533382 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3b2e3-e069-4c38-81fe-d8bb5098e5df admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:24.534122 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3b2e3-e069-4c38-81fe-d8bb5098e5df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:24.536030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3b2e3-e069-4c38-81fe-d8bb5098e5df admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:24.569396 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1383/2776] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:15:24 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 05:15:24.573574 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8ed85-28b8-46c3-bb34-422d91f77130 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:24.584997 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8ed85-28b8-46c3-bb34-422d91f77130 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:24.649831 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a2e4d-93bd-4966-b8b6-bdc59f581157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:24.650594 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a2e4d-93bd-4966-b8b6-bdc59f581157 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:24.650969 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a2e4d-93bd-4966-b8b6-bdc59f581157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:24.651402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a2e4d-93bd-4966-b8b6-bdc59f581157 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:24.652141 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a2e4d-93bd-4966-b8b6-bdc59f581157 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:24.654160 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a2e4d-93bd-4966-b8b6-bdc59f581157 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:24.678585 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1394/2777] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:15:24 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:15:24.683262 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3b2e3-e069-4c38-81fe-d8bb5098e5df admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:24.699336 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3b2e3-e069-4c38-81fe-d8bb5098e5df admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:24.706207 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f7852-21a3-4306-9b16-bb6586a096fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:24.707183 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f7852-21a3-4306-9b16-bb6586a096fc admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:24.707575 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f7852-21a3-4306-9b16-bb6586a096fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:24.707852 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f7852-21a3-4306-9b16-bb6586a096fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:24.709482 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f7852-21a3-4306-9b16-bb6586a096fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:24.711479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f7852-21a3-4306-9b16-bb6586a096fc admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:24.722535 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1384/2778] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:15:24.728134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a2e4d-93bd-4966-b8b6-bdc59f581157 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:24.748374 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a2e4d-93bd-4966-b8b6-bdc59f581157 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:24.755998 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9f69f-d1cd-40cc-89da-7f5c71cb4c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:24.756831 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9f69f-d1cd-40cc-89da-7f5c71cb4c19 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:24.757255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9f69f-d1cd-40cc-89da-7f5c71cb4c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:24.757745 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9f69f-d1cd-40cc-89da-7f5c71cb4c19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:24.758584 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9f69f-d1cd-40cc-89da-7f5c71cb4c19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:24.760514 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9f69f-d1cd-40cc-89da-7f5c71cb4c19 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:24.773361 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1395/2779] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:15:24.778327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f7852-21a3-4306-9b16-bb6586a096fc admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:24.790690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f7852-21a3-4306-9b16-bb6586a096fc admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:24.796476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188f6ac-0aad-4629-98eb-d6b4874598b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:24.797223 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188f6ac-0aad-4629-98eb-d6b4874598b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:24.797677 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188f6ac-0aad-4629-98eb-d6b4874598b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:24.798012 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188f6ac-0aad-4629-98eb-d6b4874598b4 admin admin] PATH_INFO: `/v3/projects/016110e9048a475f817a105e50f2cf6a/users/932dfc6718754b04bb50d9e8444b05a6/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:24.816211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f188f6ac-0aad-4629-98eb-d6b4874598b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=016110e9048a475f817a105e50f2cf6a, user_id=932dfc6718754b04bb50d9e8444b05a6, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:24.819110 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f188f6ac-0aad-4629-98eb-d6b4874598b4 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:24.821831 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:15:24.838277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1385/2780] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:15:24 2022] PUT /identity/v3/projects/016110e9048a475f817a105e50f2cf6a/users/932dfc6718754b04bb50d9e8444b05a6/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:15:24.843731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd455813-1f73-436e-a272-3e883171f14e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:24.844013 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd455813-1f73-436e-a272-3e883171f14e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:24.844285 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd455813-1f73-436e-a272-3e883171f14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:24.871880 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-dd455813-1f73-436e-a272-3e883171f14e None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:15:24.873873 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-dd455813-1f73-436e-a272-3e883171f14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:24.954549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1396/2781] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 05:15:24.969260 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188f6ac-0aad-4629-98eb-d6b4874598b4 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:24.971215 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-f188f6ac-0aad-4629-98eb-d6b4874598b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:25.040277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188f6ac-0aad-4629-98eb-d6b4874598b4 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:25.044199 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd72019-6da0-469d-8eab-fb2a81b39405 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:25.044818 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd72019-6da0-469d-8eab-fb2a81b39405 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:25.045130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd72019-6da0-469d-8eab-fb2a81b39405 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:25.045420 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd72019-6da0-469d-8eab-fb2a81b39405 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:25.056316 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd72019-6da0-469d-8eab-fb2a81b39405 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:25.058002 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd72019-6da0-469d-8eab-fb2a81b39405 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:25.069679 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1386/2782] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:15:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:15:30.592209 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46819490-488a-4d58-b825-a3966b9de58b None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:30.592863 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46819490-488a-4d58-b825-a3966b9de58b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:30.593207 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46819490-488a-4d58-b825-a3966b9de58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:30.651224 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-46819490-488a-4d58-b825-a3966b9de58b None None] MFA Rules not processed for user `932dfc6718754b04bb50d9e8444b05a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:15:30.653539 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-46819490-488a-4d58-b825-a3966b9de58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:30.715520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:15:30.785834 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1397/2783] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:15:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:15:30.792840 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd72019-6da0-469d-8eab-fb2a81b39405 service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:30.805628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd72019-6da0-469d-8eab-fb2a81b39405 service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:30.812485 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f6543-f036-4aa4-b2c4-3df9cfdb772a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:30.813297 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f6543-f036-4aa4-b2c4-3df9cfdb772a admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:30.813776 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f6543-f036-4aa4-b2c4-3df9cfdb772a admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:30.814670 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f6543-f036-4aa4-b2c4-3df9cfdb772a admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:30.815662 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f6543-f036-4aa4-b2c4-3df9cfdb772a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:30.818783 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f6543-f036-4aa4-b2c4-3df9cfdb772a admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:30.859296 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1387/2784] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:15:30 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 05:15:30.865022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46819490-488a-4d58-b825-a3966b9de58b None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:30.885319 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46819490-488a-4d58-b825-a3966b9de58b None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:30.982870 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db203f-f31d-412e-ba9e-b52539a044a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:30.984026 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db203f-f31d-412e-ba9e-b52539a044a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:30.984402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db203f-f31d-412e-ba9e-b52539a044a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:30.984863 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db203f-f31d-412e-ba9e-b52539a044a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:30.986109 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db203f-f31d-412e-ba9e-b52539a044a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:30.988622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db203f-f31d-412e-ba9e-b52539a044a9 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:31.017526 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1398/2785] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:15:30 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:15:31.022944 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f6543-f036-4aa4-b2c4-3df9cfdb772a admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:31.048821 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f6543-f036-4aa4-b2c4-3df9cfdb772a admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:31.054945 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2dd48e-65bd-4388-b735-565bd8a710cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:31.055748 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2dd48e-65bd-4388-b735-565bd8a710cb admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:31.056030 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2dd48e-65bd-4388-b735-565bd8a710cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:31.056507 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2dd48e-65bd-4388-b735-565bd8a710cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:31.057702 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2dd48e-65bd-4388-b735-565bd8a710cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:31.060119 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2dd48e-65bd-4388-b735-565bd8a710cb admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:31.084426 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1388/2786] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 05:15:31.088773 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db203f-f31d-412e-ba9e-b52539a044a9 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:31.105239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db203f-f31d-412e-ba9e-b52539a044a9 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:31.112493 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a1cad-47a4-4794-84fc-089af0714f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:31.113213 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a1cad-47a4-4794-84fc-089af0714f03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:31.113496 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a1cad-47a4-4794-84fc-089af0714f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:31.113798 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a1cad-47a4-4794-84fc-089af0714f03 admin admin] PATH_INFO: `/v3/projects/de56b6e3620848ed89968632fa1480ae/users/ea04013edf8b4280b4191b04a009a14b/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:31.142913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a1cad-47a4-4794-84fc-089af0714f03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de56b6e3620848ed89968632fa1480ae, user_id=ea04013edf8b4280b4191b04a009a14b, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:31.145821 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a1cad-47a4-4794-84fc-089af0714f03 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:31.167564 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1399/2787] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:15:31 2022] PUT /identity/v3/projects/de56b6e3620848ed89968632fa1480ae/users/ea04013edf8b4280b4191b04a009a14b/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:15:31.173783 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2dd48e-65bd-4388-b735-565bd8a710cb admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:31.189675 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2dd48e-65bd-4388-b735-565bd8a710cb admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:31.300168 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf2429-2ad2-4aee-84d4-3004fcc03569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:31.301210 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf2429-2ad2-4aee-84d4-3004fcc03569 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:31.301772 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf2429-2ad2-4aee-84d4-3004fcc03569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:31.302479 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf2429-2ad2-4aee-84d4-3004fcc03569 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:31.303578 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf2429-2ad2-4aee-84d4-3004fcc03569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:31.305485 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf2429-2ad2-4aee-84d4-3004fcc03569 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:31.325197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1389/2788] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:15:31.331371 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a1cad-47a4-4794-84fc-089af0714f03 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:31.343755 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf2429-2ad2-4aee-84d4-3004fcc03569 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:31.350078 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a1cad-47a4-4794-84fc-089af0714f03 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:31.356783 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab40722-5cb0-4571-8449-2721084398e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:31.357520 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab40722-5cb0-4571-8449-2721084398e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:31.357929 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab40722-5cb0-4571-8449-2721084398e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:31.358137 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab40722-5cb0-4571-8449-2721084398e0 admin admin] PATH_INFO: `/v3/projects/de56b6e3620848ed89968632fa1480ae/users/ea04013edf8b4280b4191b04a009a14b/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:31.360646 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab40722-5cb0-4571-8449-2721084398e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de56b6e3620848ed89968632fa1480ae, user_id=ea04013edf8b4280b4191b04a009a14b, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:31.363488 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab40722-5cb0-4571-8449-2721084398e0 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:31.377130 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1400/2789] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:15:31 2022] PUT /identity/v3/projects/de56b6e3620848ed89968632fa1480ae/users/ea04013edf8b4280b4191b04a009a14b/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 05:15:31.450310 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf2429-2ad2-4aee-84d4-3004fcc03569 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:31.466886 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:15:31.533900 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce096a5-9feb-4b2e-82ef-d129ea289a44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:31.534641 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce096a5-9feb-4b2e-82ef-d129ea289a44 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:31.535198 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce096a5-9feb-4b2e-82ef-d129ea289a44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:31.536868 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce096a5-9feb-4b2e-82ef-d129ea289a44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:31.539217 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-8ce096a5-9feb-4b2e-82ef-d129ea289a44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:31.562996 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce096a5-9feb-4b2e-82ef-d129ea289a44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:31.565900 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce096a5-9feb-4b2e-82ef-d129ea289a44 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:31.587409 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1390/2790] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:15:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 05:15:37.786692 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eee721-65d3-4164-a4f4-420aaf342187 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:37.787522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eee721-65d3-4164-a4f4-420aaf342187 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:37.787906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eee721-65d3-4164-a4f4-420aaf342187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:37.826220 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-d7eee721-65d3-4164-a4f4-420aaf342187 None None] MFA Rules not processed for user `ea04013edf8b4280b4191b04a009a14b`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:15:37.827696 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d7eee721-65d3-4164-a4f4-420aaf342187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:37.948523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1401/2791] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:15:37 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:15:37.964813 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce096a5-9feb-4b2e-82ef-d129ea289a44 service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:37.975503 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-8ce096a5-9feb-4b2e-82ef-d129ea289a44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:38.136206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce096a5-9feb-4b2e-82ef-d129ea289a44 service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:38.147705 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23aed92-4abe-4007-92a9-0665ef814f97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:38.148439 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23aed92-4abe-4007-92a9-0665ef814f97 service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:38.148804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23aed92-4abe-4007-92a9-0665ef814f97 service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:38.149239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23aed92-4abe-4007-92a9-0665ef814f97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:38.280893 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23aed92-4abe-4007-92a9-0665ef814f97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:38.283271 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23aed92-4abe-4007-92a9-0665ef814f97 service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:38.296592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1391/2792] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:15:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:15:39.679462 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43ae5a-5a2a-48a6-b751-3b83349752ed None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:39.680052 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43ae5a-5a2a-48a6-b751-3b83349752ed None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:39.680218 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43ae5a-5a2a-48a6-b751-3b83349752ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:39.696708 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:15:39.703807 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-9f43ae5a-5a2a-48a6-b751-3b83349752ed None None] MFA Rules not processed for user `932dfc6718754b04bb50d9e8444b05a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:15:39.705792 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9f43ae5a-5a2a-48a6-b751-3b83349752ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:39.714747 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1402/2793] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:15:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 05:15:47.510582 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c504acc-7662-4359-b908-ded460be080f None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:47.511153 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c504acc-7662-4359-b908-ded460be080f None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:47.511153 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c504acc-7662-4359-b908-ded460be080f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:47.575882 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-9c504acc-7662-4359-b908-ded460be080f None None] MFA Rules not processed for user `3a721708c7e443e799349add27de8f84`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:15:47.577425 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-9c504acc-7662-4359-b908-ded460be080f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:47.595072 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=100 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:15:47.651090 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1392/2794] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Aug 4 05:15:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:15:47.727314 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43ae5a-5a2a-48a6-b751-3b83349752ed None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:15:47.727314 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9f43ae5a-5a2a-48a6-b751-3b83349752ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:15:47.819689 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43ae5a-5a2a-48a6-b751-3b83349752ed None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:15:47.826144 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3b17b-5061-4b99-a252-5fcdff51078e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1e539006166434790823b839bd745d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:15:47.826784 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3b17b-5061-4b99-a252-5fcdff51078e service swift] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:15:47.827073 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3b17b-5061-4b99-a252-5fcdff51078e service swift] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:15:47.827410 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3b17b-5061-4b99-a252-5fcdff51078e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:15:47.838946 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3b17b-5061-4b99-a252-5fcdff51078e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:15:47.840547 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3b17b-5061-4b99-a252-5fcdff51078e service swift] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:15:47.848992 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1403/2795] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Aug 4 05:15:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 05:15:57.645225 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=19 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:15:57.845489 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=41 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:16:02.023245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7fe36-05f6-4bfb-89e0-8b0d181547f4 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:16:02.023809 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7fe36-05f6-4bfb-89e0-8b0d181547f4 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:16:02.023809 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7fe36-05f6-4bfb-89e0-8b0d181547f4 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:16:02.025196 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1393/2796] 10.222.0.47 () {54 vars in 921 bytes} [Thu Aug 4 05:16:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 05:16:02.035653 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395068f-9697-4004-b04b-246032125e62 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:16:02.036142 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395068f-9697-4004-b04b-246032125e62 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:16:02.036210 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395068f-9697-4004-b04b-246032125e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:16:02.076523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-a395068f-9697-4004-b04b-246032125e62 None None] MFA Rules not processed for user `caa2901451aa4554bdfcac1bddbc620a`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:16:02.077789 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a395068f-9697-4004-b04b-246032125e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:16:02.088757 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1404/2797] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Aug 4 05:16:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:16:02.099597 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7fe36-05f6-4bfb-89e0-8b0d181547f4 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:16:02.102196 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-eac7fe36-05f6-4bfb-89e0-8b0d181547f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:16:02.164365 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7fe36-05f6-4bfb-89e0-8b0d181547f4 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:16:02.168127 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f373d-1051-4691-b07b-01872ac39fd5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b77a955866c54a43a7aaa271fefdec57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:16:02.168745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f373d-1051-4691-b07b-01872ac39fd5 service glance] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:16:02.169028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f373d-1051-4691-b07b-01872ac39fd5 service glance] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:16:02.169301 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f373d-1051-4691-b07b-01872ac39fd5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:16:02.181935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f373d-1051-4691-b07b-01872ac39fd5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:16:02.183366 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f373d-1051-4691-b07b-01872ac39fd5 service glance] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:16:02.194177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1394/2798] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:16:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:16:02.262215 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89db4ee-fdf2-44ab-9f60-9e5b9def8292 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:16:02.262500 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89db4ee-fdf2-44ab-9f60-9e5b9def8292 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:16:02.263017 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89db4ee-fdf2-44ab-9f60-9e5b9def8292 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:16:02.264363 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1405/2799] 10.222.0.47 () {54 vars in 921 bytes} [Thu Aug 4 05:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 05:16:02.268940 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4c7de-9611-483e-bd97-ec2fbe2aafb8 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:16:02.269230 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4c7de-9611-483e-bd97-ec2fbe2aafb8 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:16:02.269556 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4c7de-9611-483e-bd97-ec2fbe2aafb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:16:02.312818 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-16d4c7de-9611-483e-bd97-ec2fbe2aafb8 None None] MFA Rules not processed for user `1a2a647f97fc4d369f562e8b5a4bf68c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:16:02.314095 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-16d4c7de-9611-483e-bd97-ec2fbe2aafb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:16:02.321085 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1395/2800] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Aug 4 05:16:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:16:02.333112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89db4ee-fdf2-44ab-9f60-9e5b9def8292 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:16:02.345479 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89db4ee-fdf2-44ab-9f60-9e5b9def8292 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:16:02.349385 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686599d-4397-4312-868d-4652659dfd4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:16:02.350076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686599d-4397-4312-868d-4652659dfd4a service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:16:02.350357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686599d-4397-4312-868d-4652659dfd4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:16:02.350632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686599d-4397-4312-868d-4652659dfd4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:16:02.358903 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686599d-4397-4312-868d-4652659dfd4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:16:02.360339 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686599d-4397-4312-868d-4652659dfd4a service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:16:02.368391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1406/2801] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:16:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:16:02.454444 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd130ac0-018d-4e78-a8c9-c3963ca48b10 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:16:02.454798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd130ac0-018d-4e78-a8c9-c3963ca48b10 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:16:02.455081 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd130ac0-018d-4e78-a8c9-c3963ca48b10 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:16:02.456445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1396/2802] 10.222.0.47 () {54 vars in 921 bytes} [Thu Aug 4 05:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 05:16:02.463145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9340fe4-4f8e-47f9-9ac9-9097e616c356 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:16:02.463145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9340fe4-4f8e-47f9-9ac9-9097e616c356 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:16:02.463372 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9340fe4-4f8e-47f9-9ac9-9097e616c356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:16:02.483741 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-c9340fe4-4f8e-47f9-9ac9-9097e616c356 None None] MFA Rules not processed for user `caa2901451aa4554bdfcac1bddbc620a`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:16:02.484934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c9340fe4-4f8e-47f9-9ac9-9097e616c356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:16:02.494418 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1407/2803] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Aug 4 05:16:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 05:16:02.509754 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd130ac0-018d-4e78-a8c9-c3963ca48b10 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:16:02.520828 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd130ac0-018d-4e78-a8c9-c3963ca48b10 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:16:02.525358 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0dcdf7-d18c-4cbf-b481-ae100394659a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:16:02.526090 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0dcdf7-d18c-4cbf-b481-ae100394659a service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:16:02.526996 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0dcdf7-d18c-4cbf-b481-ae100394659a service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:16:02.527279 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0dcdf7-d18c-4cbf-b481-ae100394659a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:16:02.538275 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0dcdf7-d18c-4cbf-b481-ae100394659a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:16:02.540185 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0dcdf7-d18c-4cbf-b481-ae100394659a service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:16:02.550998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1397/2804] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:16:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:16:04.825004 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a4d20-4980-4de6-93dd-23376e712b94 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:16:04.825504 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a4d20-4980-4de6-93dd-23376e712b94 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:16:04.825665 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a4d20-4980-4de6-93dd-23376e712b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:16:04.845399 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-9e3a4d20-4980-4de6-93dd-23376e712b94 None None] MFA Rules not processed for user `3a721708c7e443e799349add27de8f84`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:16:04.847040 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9e3a4d20-4980-4de6-93dd-23376e712b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:16:04.854804 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1408/2805] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Aug 4 05:16:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:16:04.867592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0dcdf7-d18c-4cbf-b481-ae100394659a service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:16:04.878857 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0dcdf7-d18c-4cbf-b481-ae100394659a service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:16:04.885149 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e751d98b-8b9b-4fec-8b94-bb2b7a831bfd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1e539006166434790823b839bd745d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:16:04.885788 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751d98b-8b9b-4fec-8b94-bb2b7a831bfd service swift] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:16:04.886071 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751d98b-8b9b-4fec-8b94-bb2b7a831bfd service swift] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:16:04.886344 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751d98b-8b9b-4fec-8b94-bb2b7a831bfd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:16:04.894630 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e751d98b-8b9b-4fec-8b94-bb2b7a831bfd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:16:04.896190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e751d98b-8b9b-4fec-8b94-bb2b7a831bfd service swift] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:16:04.904946 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1398/2806] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Aug 4 05:16:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 05:16:14.839989 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=21 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:16:14.901692 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=47 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:16:21.062922 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a4d20-4980-4de6-93dd-23376e712b94 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:16:21.072876 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a4d20-4980-4de6-93dd-23376e712b94 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:16:21.076724 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08e30b-d5a4-491b-af2a-98adbe0a22c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:16:21.077334 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08e30b-d5a4-491b-af2a-98adbe0a22c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:16:21.077646 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08e30b-d5a4-491b-af2a-98adbe0a22c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:16:21.077918 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08e30b-d5a4-491b-af2a-98adbe0a22c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:16:21.088134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08e30b-d5a4-491b-af2a-98adbe0a22c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:16:21.089567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08e30b-d5a4-491b-af2a-98adbe0a22c5 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:16:21.100755 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1409/2807] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:16:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 05:16:25.782976 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d1016-4417-49cb-bd3f-06bef564d537 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:16:25.783451 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d1016-4417-49cb-bd3f-06bef564d537 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:16:25.783552 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d1016-4417-49cb-bd3f-06bef564d537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:16:25.805140 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-5f0d1016-4417-49cb-bd3f-06bef564d537 None None] MFA Rules not processed for user `3a721708c7e443e799349add27de8f84`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:16:25.806377 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5f0d1016-4417-49cb-bd3f-06bef564d537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:16:25.813382 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1399/2808] 10.222.0.47 () {58 vars in 967 bytes} [Thu Aug 4 05:16:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:16:25.826437 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08e30b-d5a4-491b-af2a-98adbe0a22c5 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:16:25.838173 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08e30b-d5a4-491b-af2a-98adbe0a22c5 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:16:25.843082 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b52100-c92a-42d8-962b-77a2ca3755ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1e539006166434790823b839bd745d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:16:25.843676 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b52100-c92a-42d8-962b-77a2ca3755ae service swift] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:16:25.843956 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b52100-c92a-42d8-962b-77a2ca3755ae service swift] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:16:25.844225 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b52100-c92a-42d8-962b-77a2ca3755ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:16:25.852458 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b52100-c92a-42d8-962b-77a2ca3755ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:16:25.854104 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b52100-c92a-42d8-962b-77a2ca3755ae service swift] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:16:25.862156 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1410/2809] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Aug 4 05:16:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 05:16:35.799636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:16:35.859049 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=12 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:17:08.169292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9324083-3855-414e-836e-ec7e95f1b418 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:17:08.169837 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9324083-3855-414e-836e-ec7e95f1b418 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:17:08.169947 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9324083-3855-414e-836e-ec7e95f1b418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:17:08.189104 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-a9324083-3855-414e-836e-ec7e95f1b418 None None] MFA Rules not processed for user `28bf7f4d000248c9b40c1b4e596b5d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:17:08.190330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-a9324083-3855-414e-836e-ec7e95f1b418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:17:08.247868 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1400/2810] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:17:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:17:18.240447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=25 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:17:25.258333 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e5a6f-a6b4-47b5-818d-624ec40c0205 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:17:25.258823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e5a6f-a6b4-47b5-818d-624ec40c0205 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:17:25.258922 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e5a6f-a6b4-47b5-818d-624ec40c0205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:17:25.276997 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-cd0e5a6f-a6b4-47b5-818d-624ec40c0205 None None] MFA Rules not processed for user `30c0ff27c5a84ebd8615fb4608fc3882`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:17:25.278306 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-cd0e5a6f-a6b4-47b5-818d-624ec40c0205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:17:25.345227 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1411/2811] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:17:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:17:34.234054 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9324083-3855-414e-836e-ec7e95f1b418 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:17:34.235523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-a9324083-3855-414e-836e-ec7e95f1b418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:17:34.358362 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9324083-3855-414e-836e-ec7e95f1b418 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:17:34.363546 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aca946-4373-4c35-8f32-a0aa8cb8031b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350f72f740f749f68175c1c4e8ae9d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:17:34.364157 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aca946-4373-4c35-8f32-a0aa8cb8031b service placement] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:17:34.364437 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aca946-4373-4c35-8f32-a0aa8cb8031b service placement] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:17:34.364708 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aca946-4373-4c35-8f32-a0aa8cb8031b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:17:34.367071 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-06aca946-4373-4c35-8f32-a0aa8cb8031b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:17:34.378944 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aca946-4373-4c35-8f32-a0aa8cb8031b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:17:34.380527 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aca946-4373-4c35-8f32-a0aa8cb8031b service placement] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:17:34.390861 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1401/2812] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:17:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:17:35.338589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=31 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:17:44.386476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=54 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:18:32.946756 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f232c-33c2-4c96-8b87-3246ad8d55e5 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:18:32.947363 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f232c-33c2-4c96-8b87-3246ad8d55e5 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:18:32.947363 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f232c-33c2-4c96-8b87-3246ad8d55e5 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:18:32.948782 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1412/2813] 10.222.0.47 () {54 vars in 921 bytes} [Thu Aug 4 05:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 05:18:32.955280 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76246cc3-4089-42c7-8ef4-5d92cdd67056 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:18:32.955797 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76246cc3-4089-42c7-8ef4-5d92cdd67056 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:18:32.955883 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76246cc3-4089-42c7-8ef4-5d92cdd67056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:18:32.983891 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-76246cc3-4089-42c7-8ef4-5d92cdd67056 None None] MFA Rules not processed for user `1a2a647f97fc4d369f562e8b5a4bf68c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:18:32.985143 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-76246cc3-4089-42c7-8ef4-5d92cdd67056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:18:32.993606 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1402/2814] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Aug 4 05:18:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:18:33.007229 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f232c-33c2-4c96-8b87-3246ad8d55e5 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:18:33.019089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f232c-33c2-4c96-8b87-3246ad8d55e5 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:18:33.026443 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445ad31-bae9-427f-b137-47cd3a6b35c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:18:33.027156 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445ad31-bae9-427f-b137-47cd3a6b35c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:18:33.027533 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445ad31-bae9-427f-b137-47cd3a6b35c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:18:33.027883 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445ad31-bae9-427f-b137-47cd3a6b35c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:18:33.040565 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445ad31-bae9-427f-b137-47cd3a6b35c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:18:33.042342 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445ad31-bae9-427f-b137-47cd3a6b35c4 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:18:33.054114 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1413/2815] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:18:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:18:33.158557 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17cc00-a779-4a76-a724-2b427085c375 None None] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:18:33.159028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17cc00-a779-4a76-a724-2b427085c375 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:18:33.159757 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17cc00-a779-4a76-a724-2b427085c375 None None] PATH_INFO: `/` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:18:33.162848 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1403/2816] 10.222.0.47 () {54 vars in 921 bytes} [Thu Aug 4 05:18:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 05:18:33.169168 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181680e-fd2b-450e-bbca-187aca6f7b19 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:18:33.169453 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181680e-fd2b-450e-bbca-187aca6f7b19 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:18:33.169815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181680e-fd2b-450e-bbca-187aca6f7b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:18:33.192000 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-e181680e-fd2b-450e-bbca-187aca6f7b19 None None] MFA Rules not processed for user `caa2901451aa4554bdfcac1bddbc620a`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:18:33.193924 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e181680e-fd2b-450e-bbca-187aca6f7b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:18:33.203673 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1414/2817] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Aug 4 05:18:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:18:33.215001 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17cc00-a779-4a76-a724-2b427085c375 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:18:33.227870 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17cc00-a779-4a76-a724-2b427085c375 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:18:33.231487 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573fb22-9ee7-4e5c-8c4c-af02e943219a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:18:33.232093 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573fb22-9ee7-4e5c-8c4c-af02e943219a service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:18:33.232380 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573fb22-9ee7-4e5c-8c4c-af02e943219a service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:18:33.232649 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573fb22-9ee7-4e5c-8c4c-af02e943219a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:18:33.248725 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b573fb22-9ee7-4e5c-8c4c-af02e943219a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:18:33.251694 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b573fb22-9ee7-4e5c-8c4c-af02e943219a service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:18:33.269348 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1404/2818] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:18:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:18:43.186837 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=9 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:18:43.263315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=9 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:18:47.766198 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d8fa7-43f7-46c6-9090-c456cf9c4006 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:18:47.766880 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d8fa7-43f7-46c6-9090-c456cf9c4006 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:18:47.766880 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d8fa7-43f7-46c6-9090-c456cf9c4006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:18:47.793019 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-724d8fa7-43f7-46c6-9090-c456cf9c4006 None None] MFA Rules not processed for user `3a721708c7e443e799349add27de8f84`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:18:47.795092 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-724d8fa7-43f7-46c6-9090-c456cf9c4006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:18:47.809956 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1415/2819] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Aug 4 05:18:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 05:18:47.870717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573fb22-9ee7-4e5c-8c4c-af02e943219a service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:18:47.890397 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573fb22-9ee7-4e5c-8c4c-af02e943219a service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:18:47.898866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f7f26-6656-4275-b11d-b5dca712bd7e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1e539006166434790823b839bd745d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:18:47.899749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f7f26-6656-4275-b11d-b5dca712bd7e service swift] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:18:47.900240 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f7f26-6656-4275-b11d-b5dca712bd7e service swift] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:18:47.900725 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f7f26-6656-4275-b11d-b5dca712bd7e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:18:47.916750 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f7f26-6656-4275-b11d-b5dca712bd7e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:18:47.919440 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f7f26-6656-4275-b11d-b5dca712bd7e service swift] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:18:47.930562 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1405/2820] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Aug 4 05:18:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 05:18:57.787003 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:18:57.926833 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=6 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:19:06.231169 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f225a-060c-4193-ad79-48ae0d76dc11 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:19:06.231999 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f225a-060c-4193-ad79-48ae0d76dc11 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:19:06.231999 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f225a-060c-4193-ad79-48ae0d76dc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:19:06.252518 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b69f225a-060c-4193-ad79-48ae0d76dc11 None None] MFA Rules not processed for user `3a721708c7e443e799349add27de8f84`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:19:06.254659 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b69f225a-060c-4193-ad79-48ae0d76dc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:19:06.264070 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1416/2821] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Aug 4 05:19:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 05:19:06.277635 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f7f26-6656-4275-b11d-b5dca712bd7e service swift] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:19:06.290100 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f7f26-6656-4275-b11d-b5dca712bd7e service swift] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:19:06.295714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d8b18-4736-417d-8fb1-84930bb0ce6c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1e539006166434790823b839bd745d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:19:06.296402 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d8b18-4736-417d-8fb1-84930bb0ce6c service swift] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:19:06.296736 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d8b18-4736-417d-8fb1-84930bb0ce6c service swift] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:19:06.296892 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d8b18-4736-417d-8fb1-84930bb0ce6c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:19:06.307488 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d8b18-4736-417d-8fb1-84930bb0ce6c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:19:06.309107 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d8b18-4736-417d-8fb1-84930bb0ce6c service swift] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:19:06.319696 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1406/2822] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Aug 4 05:19:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 05:19:13.217569 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f225a-060c-4193-ad79-48ae0d76dc11 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:19:13.220638 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b69f225a-060c-4193-ad79-48ae0d76dc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:19:13.236115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f225a-060c-4193-ad79-48ae0d76dc11 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:19:13.241278 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4635c-471f-43b1-bfc4-668f375638b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:19:13.242366 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4635c-471f-43b1-bfc4-668f375638b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:19:13.242689 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4635c-471f-43b1-bfc4-668f375638b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:19:13.242936 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4635c-471f-43b1-bfc4-668f375638b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:19:13.245455 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d2d4635c-471f-43b1-bfc4-668f375638b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:19:13.262373 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4635c-471f-43b1-bfc4-668f375638b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:19:13.264083 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4635c-471f-43b1-bfc4-668f375638b4 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:19:13.279256 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1417/2823] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:19:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:19:14.520207 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d8b18-4736-417d-8fb1-84930bb0ce6c service swift] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:19:14.537888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d8b18-4736-417d-8fb1-84930bb0ce6c service swift] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:19:14.543397 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff6042-098f-483f-8b7e-15720f609224 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:19:14.544334 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff6042-098f-483f-8b7e-15720f609224 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:19:14.544779 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff6042-098f-483f-8b7e-15720f609224 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:19:14.545209 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff6042-098f-483f-8b7e-15720f609224 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:19:14.548183 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-60ff6042-098f-483f-8b7e-15720f609224 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:19:14.571153 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ff6042-098f-483f-8b7e-15720f609224 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:19:14.572626 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ff6042-098f-483f-8b7e-15720f609224 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:19:14.589823 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1407/2824] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:19:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:19:23.004879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf72515-644c-4b32-bcf1-ab9adc945400 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:19:23.005397 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf72515-644c-4b32-bcf1-ab9adc945400 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:19:23.005485 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf72515-644c-4b32-bcf1-ab9adc945400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:19:23.030945 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-8cf72515-644c-4b32-bcf1-ab9adc945400 None None] MFA Rules not processed for user `3a721708c7e443e799349add27de8f84`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:19:23.032117 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8cf72515-644c-4b32-bcf1-ab9adc945400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:19:23.042901 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1418/2825] 10.222.0.47 () {58 vars in 967 bytes} [Thu Aug 4 05:19:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:19:23.060902 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff6042-098f-483f-8b7e-15720f609224 service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:19:23.079822 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff6042-098f-483f-8b7e-15720f609224 service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:19:23.086819 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f58aea-ed8d-4536-89d5-effae7db12a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1e539006166434790823b839bd745d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:19:23.087685 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f58aea-ed8d-4536-89d5-effae7db12a3 service swift] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:19:23.088052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f58aea-ed8d-4536-89d5-effae7db12a3 service swift] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:19:23.088447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f58aea-ed8d-4536-89d5-effae7db12a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:19:23.098377 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f58aea-ed8d-4536-89d5-effae7db12a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:19:23.100065 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f58aea-ed8d-4536-89d5-effae7db12a3 service swift] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:19:23.111264 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1408/2826] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Aug 4 05:19:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 05:19:28.782013 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf72515-644c-4b32-bcf1-ab9adc945400 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:19:28.784897 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8cf72515-644c-4b32-bcf1-ab9adc945400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:19:28.791051 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=13 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:19:28.864022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf72515-644c-4b32-bcf1-ab9adc945400 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:19:28.867887 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97234b6-41f4-44dd-a878-0d158b9fd707 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:19:28.868523 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97234b6-41f4-44dd-a878-0d158b9fd707 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:19:28.868815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97234b6-41f4-44dd-a878-0d158b9fd707 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:19:28.869079 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97234b6-41f4-44dd-a878-0d158b9fd707 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:19:28.870970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a97234b6-41f4-44dd-a878-0d158b9fd707 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:19:28.883540 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97234b6-41f4-44dd-a878-0d158b9fd707 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:19:28.884989 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97234b6-41f4-44dd-a878-0d158b9fd707 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:19:28.898688 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1419/2827] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:19:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 05:19:32.679998 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a656c83-5161-425a-92a5-e1ac6f604e2d None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:19:32.680671 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a656c83-5161-425a-92a5-e1ac6f604e2d None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:19:32.680671 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a656c83-5161-425a-92a5-e1ac6f604e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:19:32.687972 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=19 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:19:32.700981 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-1a656c83-5161-425a-92a5-e1ac6f604e2d None None] MFA Rules not processed for user `30c0ff27c5a84ebd8615fb4608fc3882`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:19:32.702217 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-1a656c83-5161-425a-92a5-e1ac6f604e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:19:32.709891 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1409/2828] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:19:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 05:19:36.514780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97234b6-41f4-44dd-a878-0d158b9fd707 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:19:36.516504 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a97234b6-41f4-44dd-a878-0d158b9fd707 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:19:36.536276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97234b6-41f4-44dd-a878-0d158b9fd707 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:19:36.544710 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b1bc3-ae4a-425c-8007-13fd2fa37d59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:19:36.545344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b1bc3-ae4a-425c-8007-13fd2fa37d59 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:19:36.545642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b1bc3-ae4a-425c-8007-13fd2fa37d59 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:19:36.545916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b1bc3-ae4a-425c-8007-13fd2fa37d59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:19:36.547700 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8c0b1bc3-ae4a-425c-8007-13fd2fa37d59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:19:36.561689 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b1bc3-ae4a-425c-8007-13fd2fa37d59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:19:36.563750 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b1bc3-ae4a-425c-8007-13fd2fa37d59 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:19:36.576482 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1420/2829] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:19:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 05:19:42.696223 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:19:46.571346 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=36 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:19:57.369183 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a656c83-5161-425a-92a5-e1ac6f604e2d None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:19:57.380200 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a656c83-5161-425a-92a5-e1ac6f604e2d None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:19:57.383950 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f854cf85-fff4-472d-ab01-a750088a13a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:19:57.384582 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854cf85-fff4-472d-ab01-a750088a13a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:19:57.384849 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854cf85-fff4-472d-ab01-a750088a13a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:19:57.385120 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854cf85-fff4-472d-ab01-a750088a13a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:19:57.387025 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-f854cf85-fff4-472d-ab01-a750088a13a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:19:57.400469 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f854cf85-fff4-472d-ab01-a750088a13a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:19:57.402127 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f854cf85-fff4-472d-ab01-a750088a13a5 service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:19:57.413729 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1410/2830] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:19:57 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:20:05.278129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b1bc3-ae4a-425c-8007-13fd2fa37d59 service nova] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:05.309287 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b1bc3-ae4a-425c-8007-13fd2fa37d59 service nova] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:05.316241 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246d4d3-959c-40b9-b5dd-40fa1d6150d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:05.317235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246d4d3-959c-40b9-b5dd-40fa1d6150d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:05.317837 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246d4d3-959c-40b9-b5dd-40fa1d6150d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:05.318141 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246d4d3-959c-40b9-b5dd-40fa1d6150d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:05.321478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9246d4d3-959c-40b9-b5dd-40fa1d6150d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:05.379782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f854cf85-fff4-472d-ab01-a750088a13a5 service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:05.423583 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f854cf85-fff4-472d-ab01-a750088a13a5 service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:05.428590 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246d4d3-959c-40b9-b5dd-40fa1d6150d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:05.431306 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f79f0-b85c-4e4f-9702-86ddb6854660 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa2901451aa4554bdfcac1bddbc620a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:05.431488 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246d4d3-959c-40b9-b5dd-40fa1d6150d7 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:05.432381 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f79f0-b85c-4e4f-9702-86ddb6854660 service nova] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:05.432666 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f79f0-b85c-4e4f-9702-86ddb6854660 service nova] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:05.433001 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f79f0-b85c-4e4f-9702-86ddb6854660 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:05.435233 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-957f79f0-b85c-4e4f-9702-86ddb6854660 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:05.450788 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1421/2831] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:20:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 05:20:05.451199 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f79f0-b85c-4e4f-9702-86ddb6854660 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:05.452953 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f79f0-b85c-4e4f-9702-86ddb6854660 service nova] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:05.465661 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1411/2832] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:20:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 05:20:10.438730 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246d4d3-959c-40b9-b5dd-40fa1d6150d7 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:10.442008 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9246d4d3-959c-40b9-b5dd-40fa1d6150d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:10.469522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246d4d3-959c-40b9-b5dd-40fa1d6150d7 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:10.478338 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae29c1-1cef-4615-b8d4-2e65a04977f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350f72f740f749f68175c1c4e8ae9d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:10.479379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae29c1-1cef-4615-b8d4-2e65a04977f6 service placement] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:10.479872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae29c1-1cef-4615-b8d4-2e65a04977f6 service placement] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:10.480350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae29c1-1cef-4615-b8d4-2e65a04977f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:10.485742 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a9ae29c1-1cef-4615-b8d4-2e65a04977f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:10.529446 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae29c1-1cef-4615-b8d4-2e65a04977f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:10.533917 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae29c1-1cef-4615-b8d4-2e65a04977f6 service placement] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:10.561741 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1422/2833] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:20:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:20:10.638528 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f79f0-b85c-4e4f-9702-86ddb6854660 service nova] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:10.640124 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-957f79f0-b85c-4e4f-9702-86ddb6854660 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:10.657846 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f79f0-b85c-4e4f-9702-86ddb6854660 service nova] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:10.663509 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae44e5-79c0-47ac-a860-a2b6250c2444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:10.664287 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae44e5-79c0-47ac-a860-a2b6250c2444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:10.664856 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae44e5-79c0-47ac-a860-a2b6250c2444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:10.665392 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae44e5-79c0-47ac-a860-a2b6250c2444 admin admin] PATH_INFO: `/v3/users/28bf7f4d000248c9b40c1b4e596b5d3a` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:10.666960 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae44e5-79c0-47ac-a860-a2b6250c2444 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28bf7f4d000248c9b40c1b4e596b5d3a)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:10.669290 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae44e5-79c0-47ac-a860-a2b6250c2444 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:10.754470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-00ae44e5-79c0-47ac-a860-a2b6250c2444 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28bf7f4d000248c9b40c1b4e596b5d3a'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:10.755583 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-00ae44e5-79c0-47ac-a860-a2b6250c2444 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28bf7f4d000248c9b40c1b4e596b5d3a'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:10.778639 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-00ae44e5-79c0-47ac-a860-a2b6250c2444 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28bf7f4d000248c9b40c1b4e596b5d3a'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:10.798716 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-00ae44e5-79c0-47ac-a860-a2b6250c2444 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28bf7f4d000248c9b40c1b4e596b5d3a'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:10.802244 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-00ae44e5-79c0-47ac-a860-a2b6250c2444 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28bf7f4d000248c9b40c1b4e596b5d3a'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:10.829849 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1412/2834] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:20:10 2022] DELETE /identity/v3/users/28bf7f4d000248c9b40c1b4e596b5d3a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:11.159780 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae29c1-1cef-4615-b8d4-2e65a04977f6 service placement] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:11.182411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae29c1-1cef-4615-b8d4-2e65a04977f6 service placement] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:11.283863 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b99bc-e554-46f4-a0c4-af657c9c3a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:11.285029 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b99bc-e554-46f4-a0c4-af657c9c3a5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:11.285519 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b99bc-e554-46f4-a0c4-af657c9c3a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:11.286098 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b99bc-e554-46f4-a0c4-af657c9c3a5f admin admin] PATH_INFO: `/v3/projects/c27540203cac4353859078edecd89b4d` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:11.288603 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b99bc-e554-46f4-a0c4-af657c9c3a5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c27540203cac4353859078edecd89b4d)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:11.292471 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b99bc-e554-46f4-a0c4-af657c9c3a5f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:11.379165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-528b99bc-e554-46f4-a0c4-af657c9c3a5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c27540203cac4353859078edecd89b4d', 'request_id': 'req-528b99bc-e554-46f4-a0c4-af657c9c3a5f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:11.393173 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-528b99bc-e554-46f4-a0c4-af657c9c3a5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c27540203cac4353859078edecd89b4d', 'request_id': 'req-528b99bc-e554-46f4-a0c4-af657c9c3a5f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:11.404430 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-528b99bc-e554-46f4-a0c4-af657c9c3a5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c27540203cac4353859078edecd89b4d', 'request_id': 'req-528b99bc-e554-46f4-a0c4-af657c9c3a5f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:11.422064 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-528b99bc-e554-46f4-a0c4-af657c9c3a5f admin admin] The token cache is being invalidate because project c27540203cac4353859078edecd89b4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:20:11.423008 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-528b99bc-e554-46f4-a0c4-af657c9c3a5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:11.424761 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-528b99bc-e554-46f4-a0c4-af657c9c3a5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:11.428276 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1423/2835] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:20:11 2022] DELETE /identity/v3/projects/c27540203cac4353859078edecd89b4d => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:11.438532 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad5ac7-6498-459a-9860-9384beef05ef None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:11.439043 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad5ac7-6498-459a-9860-9384beef05ef None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:11.439476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad5ac7-6498-459a-9860-9384beef05ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:11.469831 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-36ad5ac7-6498-459a-9860-9384beef05ef None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:20:11.471155 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-36ad5ac7-6498-459a-9860-9384beef05ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:11.600271 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1413/2836] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:20:11.611156 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b83ed-1063-4fed-a083-07c55e43a782 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:11.613513 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b83ed-1063-4fed-a083-07c55e43a782 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:11.614059 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b83ed-1063-4fed-a083-07c55e43a782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:11.630396 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:11.648649 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-659b83ed-1063-4fed-a083-07c55e43a782 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:20:11.649534 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-659b83ed-1063-4fed-a083-07c55e43a782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:11.660326 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1424/2837] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:20:11.664470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad5ac7-6498-459a-9860-9384beef05ef None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:11.677437 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad5ac7-6498-459a-9860-9384beef05ef None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:11.682900 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63858002-1157-4dda-924d-f1d3437d2f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:11.683822 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63858002-1157-4dda-924d-f1d3437d2f12 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:11.684102 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63858002-1157-4dda-924d-f1d3437d2f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:11.684417 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63858002-1157-4dda-924d-f1d3437d2f12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:11.685189 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63858002-1157-4dda-924d-f1d3437d2f12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 05:20:11.685492 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63858002-1157-4dda-924d-f1d3437d2f12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:11.687732 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63858002-1157-4dda-924d-f1d3437d2f12 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:11.701776 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1414/2838] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 05:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 05:20:11.708197 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b83ed-1063-4fed-a083-07c55e43a782 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:11.723093 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b83ed-1063-4fed-a083-07c55e43a782 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:11.734687 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0905b2-f75a-4992-8b34-8bed4d763873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:11.735707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0905b2-f75a-4992-8b34-8bed4d763873 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:11.736042 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0905b2-f75a-4992-8b34-8bed4d763873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:11.736478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0905b2-f75a-4992-8b34-8bed4d763873 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:11.737589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0905b2-f75a-4992-8b34-8bed4d763873 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:11.739591 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0905b2-f75a-4992-8b34-8bed4d763873 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:11.785633 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1425/2839] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:20:11 2022] POST /identity/v3/projects => generated 365 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:20:11.791241 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63858002-1157-4dda-924d-f1d3437d2f12 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:11.808381 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63858002-1157-4dda-924d-f1d3437d2f12 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:11.846530 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:11.919188 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3d4da-9a84-4f25-b192-56ede524b1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:11.920364 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3d4da-9a84-4f25-b192-56ede524b1be admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:11.920990 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3d4da-9a84-4f25-b192-56ede524b1be admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:11.922597 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3d4da-9a84-4f25-b192-56ede524b1be admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:11.924260 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea3d4da-9a84-4f25-b192-56ede524b1be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:11.928223 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea3d4da-9a84-4f25-b192-56ede524b1be admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:11.972639 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1415/2840] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:20:11 2022] POST /identity/v3/users => generated 283 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:20:11.979087 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0905b2-f75a-4992-8b34-8bed4d763873 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:12.012215 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0905b2-f75a-4992-8b34-8bed4d763873 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:12.020748 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd48d7f-56cc-4034-9953-40ff7b1b2e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:12.022031 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd48d7f-56cc-4034-9953-40ff7b1b2e7d admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:12.022527 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd48d7f-56cc-4034-9953-40ff7b1b2e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:12.023027 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd48d7f-56cc-4034-9953-40ff7b1b2e7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:12.024462 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd48d7f-56cc-4034-9953-40ff7b1b2e7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:12.027779 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd48d7f-56cc-4034-9953-40ff7b1b2e7d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:12.044233 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1426/2841] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:20:12.049876 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3d4da-9a84-4f25-b192-56ede524b1be admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:12.063127 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3d4da-9a84-4f25-b192-56ede524b1be admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:12.068510 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f848628a-919f-4147-aea7-d1f8163febe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:12.069222 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848628a-919f-4147-aea7-d1f8163febe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:12.069661 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848628a-919f-4147-aea7-d1f8163febe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:12.069941 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848628a-919f-4147-aea7-d1f8163febe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:12.070674 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f848628a-919f-4147-aea7-d1f8163febe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:12.072304 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f848628a-919f-4147-aea7-d1f8163febe1 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:12.089096 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1416/2842] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:20:12.095308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd48d7f-56cc-4034-9953-40ff7b1b2e7d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:12.108820 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd48d7f-56cc-4034-9953-40ff7b1b2e7d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:12.115472 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaaa44c-2951-485f-9be7-2c3ad9ffc45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:12.116213 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaaa44c-2951-485f-9be7-2c3ad9ffc45a admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:12.116658 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaaa44c-2951-485f-9be7-2c3ad9ffc45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:12.116934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaaa44c-2951-485f-9be7-2c3ad9ffc45a admin admin] PATH_INFO: `/v3/projects/bf3ed32b1c554bb88692de14f3dbd5c6/users/61f56f6f1e8849aba324e5ef1689e85c/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:12.145096 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaaa44c-2951-485f-9be7-2c3ad9ffc45a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf3ed32b1c554bb88692de14f3dbd5c6, user_id=61f56f6f1e8849aba324e5ef1689e85c, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:12.147821 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaaa44c-2951-485f-9be7-2c3ad9ffc45a admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:12.172221 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1427/2843] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:20:12 2022] PUT /identity/v3/projects/bf3ed32b1c554bb88692de14f3dbd5c6/users/61f56f6f1e8849aba324e5ef1689e85c/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 05:20:12.185899 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbe4a0-6729-404f-9793-073e72075477 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:12.186278 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbe4a0-6729-404f-9793-073e72075477 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:12.186752 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbe4a0-6729-404f-9793-073e72075477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:12.226347 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-6dfbe4a0-6729-404f-9793-073e72075477 None None] MFA Rules not processed for user `61f56f6f1e8849aba324e5ef1689e85c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:20:12.227983 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-6dfbe4a0-6729-404f-9793-073e72075477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:12.363262 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1417/2844] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:20:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:20:12.368211 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaaa44c-2951-485f-9be7-2c3ad9ffc45a admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:12.477434 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaaa44c-2951-485f-9be7-2c3ad9ffc45a admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:12.484394 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611ded7-43cf-459c-8044-1905015a2f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:12.485492 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611ded7-43cf-459c-8044-1905015a2f3f admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:12.486041 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611ded7-43cf-459c-8044-1905015a2f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:12.486543 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611ded7-43cf-459c-8044-1905015a2f3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:12.487843 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4611ded7-43cf-459c-8044-1905015a2f3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:12.491726 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4611ded7-43cf-459c-8044-1905015a2f3f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:12.535299 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfbe4a0-6729-404f-9793-073e72075477 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:12.538817 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-6dfbe4a0-6729-404f-9793-073e72075477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:12.544648 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1428/2845] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:20:12 2022] POST /identity/v3/projects => generated 363 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 05:20:12.549449 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611ded7-43cf-459c-8044-1905015a2f3f admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:12.667306 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611ded7-43cf-459c-8044-1905015a2f3f admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:12.681493 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d858c6-aa20-4a46-a3b7-3c7b85bcc0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:12.682254 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d858c6-aa20-4a46-a3b7-3c7b85bcc0ed admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:12.682438 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d858c6-aa20-4a46-a3b7-3c7b85bcc0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:12.682720 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d858c6-aa20-4a46-a3b7-3c7b85bcc0ed admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:12.683404 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d858c6-aa20-4a46-a3b7-3c7b85bcc0ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:12.684401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfbe4a0-6729-404f-9793-073e72075477 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:12.685786 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d858c6-aa20-4a46-a3b7-3c7b85bcc0ed admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:12.729597 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1429/2846] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:20:12 2022] POST /identity/v3/users => generated 288 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:20:12.736797 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d858c6-aa20-4a46-a3b7-3c7b85bcc0ed admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:12.748426 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d858c6-aa20-4a46-a3b7-3c7b85bcc0ed admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:12.748632 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:12.754763 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600abe17-0837-4298-b69a-74170bf19839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:12.755501 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600abe17-0837-4298-b69a-74170bf19839 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:12.755987 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600abe17-0837-4298-b69a-74170bf19839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:12.756330 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600abe17-0837-4298-b69a-74170bf19839 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:12.757132 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600abe17-0837-4298-b69a-74170bf19839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:12.758956 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600abe17-0837-4298-b69a-74170bf19839 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:12.785235 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1430/2847] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:20:12.787066 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600abe17-0837-4298-b69a-74170bf19839 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:12.799104 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:12.808231 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600abe17-0837-4298-b69a-74170bf19839 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:12.811050 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75af41b-b327-4684-b436-aadfecf9da1b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350f72f740f749f68175c1c4e8ae9d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:12.812404 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75af41b-b327-4684-b436-aadfecf9da1b service placement] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:12.814309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75af41b-b327-4684-b436-aadfecf9da1b service placement] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:12.814793 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75af41b-b327-4684-b436-aadfecf9da1b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:12.817420 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6e2c3-8f7b-40a8-99e4-5732619d9554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:12.818232 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6e2c3-8f7b-40a8-99e4-5732619d9554 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:12.818521 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6e2c3-8f7b-40a8-99e4-5732619d9554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:12.818698 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-d75af41b-b327-4684-b436-aadfecf9da1b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:12.818822 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6e2c3-8f7b-40a8-99e4-5732619d9554 admin admin] PATH_INFO: `/v3/projects/6fbf4a28abb04fa0af834c0ec8ed8656/users/ba4d6f833d2145da8cf9bd266a7d5105/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:12.844594 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75af41b-b327-4684-b436-aadfecf9da1b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:12.846664 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc6e2c3-8f7b-40a8-99e4-5732619d9554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fbf4a28abb04fa0af834c0ec8ed8656, user_id=ba4d6f833d2145da8cf9bd266a7d5105, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:12.847695 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75af41b-b327-4684-b436-aadfecf9da1b service placement] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:12.849376 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc6e2c3-8f7b-40a8-99e4-5732619d9554 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:12.874822 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1431/2848] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:20:12 2022] PUT /identity/v3/projects/6fbf4a28abb04fa0af834c0ec8ed8656/users/ba4d6f833d2145da8cf9bd266a7d5105/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:12.875991 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1418/2849] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:20:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:20:12.879128 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6e2c3-8f7b-40a8-99e4-5732619d9554 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:13.012167 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6e2c3-8f7b-40a8-99e4-5732619d9554 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:13.020391 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e2f55b-1db3-46e2-9a81-7663577a25e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:13.021045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2f55b-1db3-46e2-9a81-7663577a25e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:13.021329 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2f55b-1db3-46e2-9a81-7663577a25e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:13.021646 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2f55b-1db3-46e2-9a81-7663577a25e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:13.022390 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e2f55b-1db3-46e2-9a81-7663577a25e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:13.024103 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e2f55b-1db3-46e2-9a81-7663577a25e8 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:13.039417 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1432/2850] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:20:13.045748 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75af41b-b327-4684-b436-aadfecf9da1b service placement] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:13.064121 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75af41b-b327-4684-b436-aadfecf9da1b service placement] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:13.070217 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4669ed0-c6f7-4dbc-b816-9eaafe8b0880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:13.070939 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4669ed0-c6f7-4dbc-b816-9eaafe8b0880 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:13.071219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4669ed0-c6f7-4dbc-b816-9eaafe8b0880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:13.071489 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4669ed0-c6f7-4dbc-b816-9eaafe8b0880 admin admin] PATH_INFO: `/v3/projects/6fbf4a28abb04fa0af834c0ec8ed8656/users/ba4d6f833d2145da8cf9bd266a7d5105/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:13.073930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4669ed0-c6f7-4dbc-b816-9eaafe8b0880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fbf4a28abb04fa0af834c0ec8ed8656, user_id=ba4d6f833d2145da8cf9bd266a7d5105, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:13.076616 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4669ed0-c6f7-4dbc-b816-9eaafe8b0880 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:13.090103 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1419/2851] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:20:13 2022] PUT /identity/v3/projects/6fbf4a28abb04fa0af834c0ec8ed8656/users/ba4d6f833d2145da8cf9bd266a7d5105/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:13.103169 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a2eb4-b681-4e57-a301-ef5d3e428ab8 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:13.103655 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a2eb4-b681-4e57-a301-ef5d3e428ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:13.104112 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a2eb4-b681-4e57-a301-ef5d3e428ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:13.150815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-8b2a2eb4-b681-4e57-a301-ef5d3e428ab8 None None] MFA Rules not processed for user `ba4d6f833d2145da8cf9bd266a7d5105`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:20:13.152046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8b2a2eb4-b681-4e57-a301-ef5d3e428ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:13.259361 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1433/2852] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:20:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 05:20:13.277449 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4669ed0-c6f7-4dbc-b816-9eaafe8b0880 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:13.279740 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-e4669ed0-c6f7-4dbc-b816-9eaafe8b0880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:13.386838 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4669ed0-c6f7-4dbc-b816-9eaafe8b0880 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:13.393714 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c09ee1-5431-4329-95ca-f9ea7c6d7f92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:13.394382 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c09ee1-5431-4329-95ca-f9ea7c6d7f92 service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:13.394656 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c09ee1-5431-4329-95ca-f9ea7c6d7f92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:13.394930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c09ee1-5431-4329-95ca-f9ea7c6d7f92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:13.407523 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c09ee1-5431-4329-95ca-f9ea7c6d7f92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:13.410592 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c09ee1-5431-4329-95ca-f9ea7c6d7f92 service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:13.429111 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1420/2853] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:20:13 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:20:13.548861 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a2eb4-b681-4e57-a301-ef5d3e428ab8 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:13.560582 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a2eb4-b681-4e57-a301-ef5d3e428ab8 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:13.566057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee57c5-6330-4d72-ac79-0bace5488642 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:13.566823 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee57c5-6330-4d72-ac79-0bace5488642 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:13.567210 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee57c5-6330-4d72-ac79-0bace5488642 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:13.567582 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee57c5-6330-4d72-ac79-0bace5488642 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:13.654758 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee57c5-6330-4d72-ac79-0bace5488642 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:13.658538 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee57c5-6330-4d72-ac79-0bace5488642 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:13.667615 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:13.690056 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1434/2854] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:20:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 05:20:17.164465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c09ee1-5431-4329-95ca-f9ea7c6d7f92 service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:17.166838 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-03c09ee1-5431-4329-95ca-f9ea7c6d7f92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:17.280029 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c09ee1-5431-4329-95ca-f9ea7c6d7f92 service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:17.285681 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a4acc-446b-4c81-ba0b-e5aa0bcd6d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:17.286337 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a4acc-446b-4c81-ba0b-e5aa0bcd6d8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:17.286620 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a4acc-446b-4c81-ba0b-e5aa0bcd6d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:17.286885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a4acc-446b-4c81-ba0b-e5aa0bcd6d8f admin admin] PATH_INFO: `/v3/users/932dfc6718754b04bb50d9e8444b05a6` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:17.288327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a4acc-446b-4c81-ba0b-e5aa0bcd6d8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=932dfc6718754b04bb50d9e8444b05a6)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:17.290327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a4acc-446b-4c81-ba0b-e5aa0bcd6d8f admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:17.368128 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-894a4acc-446b-4c81-ba0b-e5aa0bcd6d8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '932dfc6718754b04bb50d9e8444b05a6'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:17.369405 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-894a4acc-446b-4c81-ba0b-e5aa0bcd6d8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '932dfc6718754b04bb50d9e8444b05a6'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:17.373708 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:17.400933 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-894a4acc-446b-4c81-ba0b-e5aa0bcd6d8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '932dfc6718754b04bb50d9e8444b05a6'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:17.411690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-894a4acc-446b-4c81-ba0b-e5aa0bcd6d8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '932dfc6718754b04bb50d9e8444b05a6'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:17.412340 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-894a4acc-446b-4c81-ba0b-e5aa0bcd6d8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '932dfc6718754b04bb50d9e8444b05a6'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:17.435303 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1421/2855] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:20:17 2022] DELETE /identity/v3/users/932dfc6718754b04bb50d9e8444b05a6 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:17.725476 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee57c5-6330-4d72-ac79-0bace5488642 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:17.742564 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee57c5-6330-4d72-ac79-0bace5488642 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:17.854905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbc26e-d9e5-4a33-b3fd-839b0e642399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:17.855567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbc26e-d9e5-4a33-b3fd-839b0e642399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:17.855869 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbc26e-d9e5-4a33-b3fd-839b0e642399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:17.856131 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbc26e-d9e5-4a33-b3fd-839b0e642399 admin admin] PATH_INFO: `/v3/projects/016110e9048a475f817a105e50f2cf6a` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:17.857495 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bbc26e-d9e5-4a33-b3fd-839b0e642399 admin admin] RBAC: Authorizing `identity:delete_project(project_id=016110e9048a475f817a105e50f2cf6a)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:17.860185 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bbc26e-d9e5-4a33-b3fd-839b0e642399 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:17.926538 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-21bbc26e-d9e5-4a33-b3fd-839b0e642399 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '016110e9048a475f817a105e50f2cf6a', 'request_id': 'req-21bbc26e-d9e5-4a33-b3fd-839b0e642399'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:17.935134 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-21bbc26e-d9e5-4a33-b3fd-839b0e642399 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '016110e9048a475f817a105e50f2cf6a', 'request_id': 'req-21bbc26e-d9e5-4a33-b3fd-839b0e642399'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:17.952152 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-21bbc26e-d9e5-4a33-b3fd-839b0e642399 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '016110e9048a475f817a105e50f2cf6a', 'request_id': 'req-21bbc26e-d9e5-4a33-b3fd-839b0e642399'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:17.966384 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-21bbc26e-d9e5-4a33-b3fd-839b0e642399 admin admin] The token cache is being invalidate because project 016110e9048a475f817a105e50f2cf6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:20:17.966675 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-21bbc26e-d9e5-4a33-b3fd-839b0e642399 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:17.967392 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-21bbc26e-d9e5-4a33-b3fd-839b0e642399 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:17.969123 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1435/2856] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:20:17 2022] DELETE /identity/v3/projects/016110e9048a475f817a105e50f2cf6a => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:17.974996 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a4acc-446b-4c81-ba0b-e5aa0bcd6d8f admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:17.977203 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-894a4acc-446b-4c81-ba0b-e5aa0bcd6d8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:17.999975 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a4acc-446b-4c81-ba0b-e5aa0bcd6d8f admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:18.096139 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db481a90-9b6f-4aa3-bd9a-39483ec965cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:18.097003 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db481a90-9b6f-4aa3-bd9a-39483ec965cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:18.097409 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db481a90-9b6f-4aa3-bd9a-39483ec965cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:18.097917 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db481a90-9b6f-4aa3-bd9a-39483ec965cb admin admin] PATH_INFO: `/v3/users/ea04013edf8b4280b4191b04a009a14b` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:18.099886 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db481a90-9b6f-4aa3-bd9a-39483ec965cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea04013edf8b4280b4191b04a009a14b)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:18.103057 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db481a90-9b6f-4aa3-bd9a-39483ec965cb admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:18.168106 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-db481a90-9b6f-4aa3-bd9a-39483ec965cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea04013edf8b4280b4191b04a009a14b'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:18.168839 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-db481a90-9b6f-4aa3-bd9a-39483ec965cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea04013edf8b4280b4191b04a009a14b'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:18.186197 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-db481a90-9b6f-4aa3-bd9a-39483ec965cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea04013edf8b4280b4191b04a009a14b'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:18.197212 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-db481a90-9b6f-4aa3-bd9a-39483ec965cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea04013edf8b4280b4191b04a009a14b'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:18.197993 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-db481a90-9b6f-4aa3-bd9a-39483ec965cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea04013edf8b4280b4191b04a009a14b'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:18.219706 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1422/2857] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:20:17 2022] DELETE /identity/v3/users/ea04013edf8b4280b4191b04a009a14b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:18.465206 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbc26e-d9e5-4a33-b3fd-839b0e642399 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:18.466779 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-21bbc26e-d9e5-4a33-b3fd-839b0e642399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:18.486894 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbc26e-d9e5-4a33-b3fd-839b0e642399 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:18.574098 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a970d8-0f29-4c31-988b-1dc7b2bfe4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:18.575215 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a970d8-0f29-4c31-988b-1dc7b2bfe4a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:18.575716 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a970d8-0f29-4c31-988b-1dc7b2bfe4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:18.576224 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a970d8-0f29-4c31-988b-1dc7b2bfe4a9 admin admin] PATH_INFO: `/v3/projects/de56b6e3620848ed89968632fa1480ae` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:18.579069 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a970d8-0f29-4c31-988b-1dc7b2bfe4a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de56b6e3620848ed89968632fa1480ae)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:18.582877 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a970d8-0f29-4c31-988b-1dc7b2bfe4a9 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:18.639645 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:18.656758 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-71a970d8-0f29-4c31-988b-1dc7b2bfe4a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de56b6e3620848ed89968632fa1480ae', 'request_id': 'req-71a970d8-0f29-4c31-988b-1dc7b2bfe4a9'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:18.666894 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-71a970d8-0f29-4c31-988b-1dc7b2bfe4a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de56b6e3620848ed89968632fa1480ae', 'request_id': 'req-71a970d8-0f29-4c31-988b-1dc7b2bfe4a9'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:18.673897 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-71a970d8-0f29-4c31-988b-1dc7b2bfe4a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de56b6e3620848ed89968632fa1480ae', 'request_id': 'req-71a970d8-0f29-4c31-988b-1dc7b2bfe4a9'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:18.686002 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-71a970d8-0f29-4c31-988b-1dc7b2bfe4a9 admin admin] The token cache is being invalidate because project de56b6e3620848ed89968632fa1480ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:20:18.686586 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-71a970d8-0f29-4c31-988b-1dc7b2bfe4a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:18.687358 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-71a970d8-0f29-4c31-988b-1dc7b2bfe4a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:18.688925 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1436/2858] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:20:18 2022] DELETE /identity/v3/projects/de56b6e3620848ed89968632fa1480ae => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:24.116735 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db481a90-9b6f-4aa3-bd9a-39483ec965cb admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:24.118343 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-db481a90-9b6f-4aa3-bd9a-39483ec965cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:24.134245 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db481a90-9b6f-4aa3-bd9a-39483ec965cb admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:24.200097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:24.217659 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a49268-35ad-4edd-9956-e0fc9dffef49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:24.218925 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a49268-35ad-4edd-9956-e0fc9dffef49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:24.219205 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a49268-35ad-4edd-9956-e0fc9dffef49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:24.219544 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a49268-35ad-4edd-9956-e0fc9dffef49 admin admin] PATH_INFO: `/v3/users/61f56f6f1e8849aba324e5ef1689e85c` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:24.221292 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a49268-35ad-4edd-9956-e0fc9dffef49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61f56f6f1e8849aba324e5ef1689e85c)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:24.224671 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a49268-35ad-4edd-9956-e0fc9dffef49 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:24.299940 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-88a49268-35ad-4edd-9956-e0fc9dffef49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61f56f6f1e8849aba324e5ef1689e85c'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:24.301025 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-88a49268-35ad-4edd-9956-e0fc9dffef49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61f56f6f1e8849aba324e5ef1689e85c'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:24.323387 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-88a49268-35ad-4edd-9956-e0fc9dffef49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61f56f6f1e8849aba324e5ef1689e85c'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:24.336867 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-88a49268-35ad-4edd-9956-e0fc9dffef49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61f56f6f1e8849aba324e5ef1689e85c'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:24.338001 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-88a49268-35ad-4edd-9956-e0fc9dffef49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61f56f6f1e8849aba324e5ef1689e85c'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:24.357431 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1423/2859] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:20:24 2022] DELETE /identity/v3/users/61f56f6f1e8849aba324e5ef1689e85c => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:24.364625 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036c946-44bf-4e37-9407-8099f95bf248 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:24.365483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036c946-44bf-4e37-9407-8099f95bf248 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:24.365483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036c946-44bf-4e37-9407-8099f95bf248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:24.401458 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b036c946-44bf-4e37-9407-8099f95bf248 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:20:24.402632 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b036c946-44bf-4e37-9407-8099f95bf248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:24.495573 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1437/2860] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 05:20:24.508964 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a49268-35ad-4edd-9956-e0fc9dffef49 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:24.510019 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-88a49268-35ad-4edd-9956-e0fc9dffef49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:24.640444 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a49268-35ad-4edd-9956-e0fc9dffef49 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:24.645070 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47959c6-2714-490a-83b0-f1de7508a63f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:24.646033 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47959c6-2714-490a-83b0-f1de7508a63f service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:24.646664 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47959c6-2714-490a-83b0-f1de7508a63f service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:24.646990 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47959c6-2714-490a-83b0-f1de7508a63f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:24.660970 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47959c6-2714-490a-83b0-f1de7508a63f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:24.662612 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47959c6-2714-490a-83b0-f1de7508a63f service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:24.678755 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1424/2861] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:20:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:20:25.050787 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036c946-44bf-4e37-9407-8099f95bf248 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:25.052961 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b036c946-44bf-4e37-9407-8099f95bf248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:25.072519 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036c946-44bf-4e37-9407-8099f95bf248 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:25.077864 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31692abf-f3b7-46ad-97a0-ba8fcbfdda78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:25.078531 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31692abf-f3b7-46ad-97a0-ba8fcbfdda78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:25.078813 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31692abf-f3b7-46ad-97a0-ba8fcbfdda78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:25.079092 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31692abf-f3b7-46ad-97a0-ba8fcbfdda78 admin admin] PATH_INFO: `/v3/projects/bf3ed32b1c554bb88692de14f3dbd5c6` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:25.080345 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31692abf-f3b7-46ad-97a0-ba8fcbfdda78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf3ed32b1c554bb88692de14f3dbd5c6)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:25.082367 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31692abf-f3b7-46ad-97a0-ba8fcbfdda78 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:25.148620 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-31692abf-f3b7-46ad-97a0-ba8fcbfdda78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf3ed32b1c554bb88692de14f3dbd5c6', 'request_id': 'req-31692abf-f3b7-46ad-97a0-ba8fcbfdda78'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:25.161186 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-31692abf-f3b7-46ad-97a0-ba8fcbfdda78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf3ed32b1c554bb88692de14f3dbd5c6', 'request_id': 'req-31692abf-f3b7-46ad-97a0-ba8fcbfdda78'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:25.168940 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-31692abf-f3b7-46ad-97a0-ba8fcbfdda78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf3ed32b1c554bb88692de14f3dbd5c6', 'request_id': 'req-31692abf-f3b7-46ad-97a0-ba8fcbfdda78'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:25.185129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-31692abf-f3b7-46ad-97a0-ba8fcbfdda78 admin admin] The token cache is being invalidate because project bf3ed32b1c554bb88692de14f3dbd5c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:20:25.185905 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-31692abf-f3b7-46ad-97a0-ba8fcbfdda78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:25.187526 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-31692abf-f3b7-46ad-97a0-ba8fcbfdda78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:25.189972 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1438/2862] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:20:25 2022] DELETE /identity/v3/projects/bf3ed32b1c554bb88692de14f3dbd5c6 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:25.198671 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47959c6-2714-490a-83b0-f1de7508a63f service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:25.210897 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47959c6-2714-490a-83b0-f1de7508a63f service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:25.287004 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:25.298491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf32f7-9c9e-48a6-b3b5-3b9fa31dc20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:25.298491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf32f7-9c9e-48a6-b3b5-3b9fa31dc20e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:25.298491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf32f7-9c9e-48a6-b3b5-3b9fa31dc20e admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:25.298491 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf32f7-9c9e-48a6-b3b5-3b9fa31dc20e admin admin] PATH_INFO: `/v3/users/ba4d6f833d2145da8cf9bd266a7d5105` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:25.301000 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf32f7-9c9e-48a6-b3b5-3b9fa31dc20e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba4d6f833d2145da8cf9bd266a7d5105)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:25.303147 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf32f7-9c9e-48a6-b3b5-3b9fa31dc20e admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:25.369353 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d1cf32f7-9c9e-48a6-b3b5-3b9fa31dc20e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba4d6f833d2145da8cf9bd266a7d5105'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:25.370279 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d1cf32f7-9c9e-48a6-b3b5-3b9fa31dc20e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba4d6f833d2145da8cf9bd266a7d5105'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:25.386912 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d1cf32f7-9c9e-48a6-b3b5-3b9fa31dc20e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba4d6f833d2145da8cf9bd266a7d5105'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:25.399898 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d1cf32f7-9c9e-48a6-b3b5-3b9fa31dc20e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba4d6f833d2145da8cf9bd266a7d5105'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:25.401150 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-d1cf32f7-9c9e-48a6-b3b5-3b9fa31dc20e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba4d6f833d2145da8cf9bd266a7d5105'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:25.428954 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1425/2863] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:20:25 2022] DELETE /identity/v3/users/ba4d6f833d2145da8cf9bd266a7d5105 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:25.882288 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31692abf-f3b7-46ad-97a0-ba8fcbfdda78 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:25.884531 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-31692abf-f3b7-46ad-97a0-ba8fcbfdda78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:25.911721 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31692abf-f3b7-46ad-97a0-ba8fcbfdda78 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:25.961731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:25.999799 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8b31a-0ee3-484b-9938-dfb7da202ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:26.000841 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8b31a-0ee3-484b-9938-dfb7da202ca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:26.001057 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8b31a-0ee3-484b-9938-dfb7da202ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:26.001342 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8b31a-0ee3-484b-9938-dfb7da202ca3 admin admin] PATH_INFO: `/v3/projects/6fbf4a28abb04fa0af834c0ec8ed8656` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:26.002676 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8b31a-0ee3-484b-9938-dfb7da202ca3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fbf4a28abb04fa0af834c0ec8ed8656)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:26.004897 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8b31a-0ee3-484b-9938-dfb7da202ca3 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:26.074037 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4ba8b31a-0ee3-484b-9938-dfb7da202ca3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fbf4a28abb04fa0af834c0ec8ed8656', 'request_id': 'req-4ba8b31a-0ee3-484b-9938-dfb7da202ca3'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:26.086536 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4ba8b31a-0ee3-484b-9938-dfb7da202ca3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fbf4a28abb04fa0af834c0ec8ed8656', 'request_id': 'req-4ba8b31a-0ee3-484b-9938-dfb7da202ca3'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:26.096934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4ba8b31a-0ee3-484b-9938-dfb7da202ca3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fbf4a28abb04fa0af834c0ec8ed8656', 'request_id': 'req-4ba8b31a-0ee3-484b-9938-dfb7da202ca3'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:26.117169 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4ba8b31a-0ee3-484b-9938-dfb7da202ca3 admin admin] The token cache is being invalidate because project 6fbf4a28abb04fa0af834c0ec8ed8656 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:20:26.117681 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4ba8b31a-0ee3-484b-9938-dfb7da202ca3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:26.118802 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4ba8b31a-0ee3-484b-9938-dfb7da202ca3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:26.121246 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1439/2864] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:20:25 2022] DELETE /identity/v3/projects/6fbf4a28abb04fa0af834c0ec8ed8656 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:26.129269 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ab37d-dcc2-4b06-a414-cf42a12b4c06 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:26.129787 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ab37d-dcc2-4b06-a414-cf42a12b4c06 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:26.130211 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ab37d-dcc2-4b06-a414-cf42a12b4c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:26.162445 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-397ab37d-dcc2-4b06-a414-cf42a12b4c06 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:20:26.164354 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-397ab37d-dcc2-4b06-a414-cf42a12b4c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:26.256077 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1426/2865] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:20:26.268644 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d2947-8dc8-4d44-b5fd-5c245bcaa249 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:26.268942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d2947-8dc8-4d44-b5fd-5c245bcaa249 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:26.269460 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d2947-8dc8-4d44-b5fd-5c245bcaa249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:26.292872 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-0e3d2947-8dc8-4d44-b5fd-5c245bcaa249 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:20:26.294165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0e3d2947-8dc8-4d44-b5fd-5c245bcaa249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:26.303311 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1440/2866] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:20:26.310311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ab37d-dcc2-4b06-a414-cf42a12b4c06 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:26.329707 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ab37d-dcc2-4b06-a414-cf42a12b4c06 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:26.337174 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c8bee-6370-4786-9ee6-bfe7f9ee2895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:26.338199 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c8bee-6370-4786-9ee6-bfe7f9ee2895 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:26.338643 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c8bee-6370-4786-9ee6-bfe7f9ee2895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:26.339082 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c8bee-6370-4786-9ee6-bfe7f9ee2895 admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:26.340222 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48c8bee-6370-4786-9ee6-bfe7f9ee2895 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 05:20:26.340711 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48c8bee-6370-4786-9ee6-bfe7f9ee2895 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:26.344023 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48c8bee-6370-4786-9ee6-bfe7f9ee2895 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:26.366367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1427/2867] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 05:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 05:20:26.370957 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d2947-8dc8-4d44-b5fd-5c245bcaa249 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:26.382396 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d2947-8dc8-4d44-b5fd-5c245bcaa249 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:26.392738 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6bc3f-9b31-47cb-a5f4-eb5e65e202ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:26.393916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6bc3f-9b31-47cb-a5f4-eb5e65e202ba admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:26.394405 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6bc3f-9b31-47cb-a5f4-eb5e65e202ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:26.394879 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6bc3f-9b31-47cb-a5f4-eb5e65e202ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:26.396205 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6bc3f-9b31-47cb-a5f4-eb5e65e202ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:26.400158 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6bc3f-9b31-47cb-a5f4-eb5e65e202ba admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:26.449955 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1441/2868] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:20:26 2022] POST /identity/v3/projects => generated 387 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:20:26.455906 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c8bee-6370-4786-9ee6-bfe7f9ee2895 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:26.476823 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c8bee-6370-4786-9ee6-bfe7f9ee2895 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:26.554621 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240a73c-a24f-4f08-bfc8-3b87b9319327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:26.555288 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240a73c-a24f-4f08-bfc8-3b87b9319327 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:26.555639 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240a73c-a24f-4f08-bfc8-3b87b9319327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:26.555955 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240a73c-a24f-4f08-bfc8-3b87b9319327 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:26.556724 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5240a73c-a24f-4f08-bfc8-3b87b9319327 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:26.558940 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5240a73c-a24f-4f08-bfc8-3b87b9319327 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:26.569682 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:26.587243 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1428/2869] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:20:26 2022] POST /identity/v3/users => generated 294 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:20:26.591928 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6bc3f-9b31-47cb-a5f4-eb5e65e202ba admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:26.604223 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6bc3f-9b31-47cb-a5f4-eb5e65e202ba admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:26.612035 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01a8a4-882f-4479-9e6d-022d7aad7066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:26.613089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01a8a4-882f-4479-9e6d-022d7aad7066 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:26.613622 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01a8a4-882f-4479-9e6d-022d7aad7066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:26.614100 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01a8a4-882f-4479-9e6d-022d7aad7066 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:26.615371 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01a8a4-882f-4479-9e6d-022d7aad7066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:26.618426 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01a8a4-882f-4479-9e6d-022d7aad7066 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:26.636572 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1442/2870] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:20:26.643267 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240a73c-a24f-4f08-bfc8-3b87b9319327 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:26.662937 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240a73c-a24f-4f08-bfc8-3b87b9319327 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:26.671967 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407186f8-93c0-4f48-a4ef-73c8fed2b562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:26.672887 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407186f8-93c0-4f48-a4ef-73c8fed2b562 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:26.673248 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407186f8-93c0-4f48-a4ef-73c8fed2b562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:26.673706 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407186f8-93c0-4f48-a4ef-73c8fed2b562 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:26.674529 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407186f8-93c0-4f48-a4ef-73c8fed2b562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:26.676497 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407186f8-93c0-4f48-a4ef-73c8fed2b562 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:26.692770 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1429/2871] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:20:26.699049 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01a8a4-882f-4479-9e6d-022d7aad7066 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:26.711589 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01a8a4-882f-4479-9e6d-022d7aad7066 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:26.716793 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed709d9-17d4-4df4-aabe-f23b527aac38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:26.718294 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed709d9-17d4-4df4-aabe-f23b527aac38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:26.718754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed709d9-17d4-4df4-aabe-f23b527aac38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:26.719217 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed709d9-17d4-4df4-aabe-f23b527aac38 admin admin] PATH_INFO: `/v3/projects/2e50f256740c447a821e0ca76787cc3a/users/0e0c6e41740944d7ab5087cdfc444e70/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:26.738351 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed709d9-17d4-4df4-aabe-f23b527aac38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e50f256740c447a821e0ca76787cc3a, user_id=0e0c6e41740944d7ab5087cdfc444e70, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:26.740990 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed709d9-17d4-4df4-aabe-f23b527aac38 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:26.762538 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1443/2872] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:20:26 2022] PUT /identity/v3/projects/2e50f256740c447a821e0ca76787cc3a/users/0e0c6e41740944d7ab5087cdfc444e70/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:26.779697 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01684772-1d9c-409b-a1df-f9026ad8339f None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:26.780907 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01684772-1d9c-409b-a1df-f9026ad8339f None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:26.781418 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01684772-1d9c-409b-a1df-f9026ad8339f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:26.829930 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-01684772-1d9c-409b-a1df-f9026ad8339f None None] MFA Rules not processed for user `0e0c6e41740944d7ab5087cdfc444e70`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:20:26.832019 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-01684772-1d9c-409b-a1df-f9026ad8339f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:26.928315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1430/2873] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:20:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:20:26.935289 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed709d9-17d4-4df4-aabe-f23b527aac38 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:27.012379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:27.017325 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed709d9-17d4-4df4-aabe-f23b527aac38 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:27.025687 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd682a0-0336-41f3-8495-8ed574b05506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:27.026791 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd682a0-0336-41f3-8495-8ed574b05506 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:27.027187 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd682a0-0336-41f3-8495-8ed574b05506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:27.027736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd682a0-0336-41f3-8495-8ed574b05506 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:27.028548 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd682a0-0336-41f3-8495-8ed574b05506 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:27.031483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd682a0-0336-41f3-8495-8ed574b05506 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:27.060517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1444/2874] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:20:26 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:20:27.069833 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01684772-1d9c-409b-a1df-f9026ad8339f None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:27.084287 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01684772-1d9c-409b-a1df-f9026ad8339f None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:27.169251 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd8b62-f055-4da9-a20f-d644b085eed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:27.169937 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd8b62-f055-4da9-a20f-d644b085eed9 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:27.170329 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd8b62-f055-4da9-a20f-d644b085eed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:27.170642 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd8b62-f055-4da9-a20f-d644b085eed9 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:27.171318 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd8b62-f055-4da9-a20f-d644b085eed9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:27.173369 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd8b62-f055-4da9-a20f-d644b085eed9 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:27.199474 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1431/2875] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:20:27 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:20:27.207698 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd682a0-0336-41f3-8495-8ed574b05506 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:27.225984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd682a0-0336-41f3-8495-8ed574b05506 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:27.231896 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7258f81-c872-4e20-a2b6-826cb4c24584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:27.232709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7258f81-c872-4e20-a2b6-826cb4c24584 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:27.233228 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7258f81-c872-4e20-a2b6-826cb4c24584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:27.233713 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7258f81-c872-4e20-a2b6-826cb4c24584 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:27.234546 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7258f81-c872-4e20-a2b6-826cb4c24584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:27.236876 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7258f81-c872-4e20-a2b6-826cb4c24584 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:27.250756 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1445/2876] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:20:27.255831 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd8b62-f055-4da9-a20f-d644b085eed9 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:27.269880 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd8b62-f055-4da9-a20f-d644b085eed9 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:27.278324 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3cc19-fb9e-4dc1-8ac9-6f881f6d15a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:27.279163 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3cc19-fb9e-4dc1-8ac9-6f881f6d15a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:27.279545 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3cc19-fb9e-4dc1-8ac9-6f881f6d15a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:27.280064 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3cc19-fb9e-4dc1-8ac9-6f881f6d15a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:27.280933 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3cc19-fb9e-4dc1-8ac9-6f881f6d15a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:27.282937 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3cc19-fb9e-4dc1-8ac9-6f881f6d15a6 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:27.298016 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1432/2877] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 05:20:27.299814 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7258f81-c872-4e20-a2b6-826cb4c24584 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:27.312896 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7258f81-c872-4e20-a2b6-826cb4c24584 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:27.318774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5a0f2-a941-450c-8386-f24425d7572f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:27.319827 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5a0f2-a941-450c-8386-f24425d7572f admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:27.320275 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5a0f2-a941-450c-8386-f24425d7572f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:27.320695 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5a0f2-a941-450c-8386-f24425d7572f admin admin] PATH_INFO: `/v3/projects/9121c63f41da43828687fd8e01a18963/users/dcc68c9d4c274f16946d8fb2e0a502bd/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:27.345709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5a0f2-a941-450c-8386-f24425d7572f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9121c63f41da43828687fd8e01a18963, user_id=dcc68c9d4c274f16946d8fb2e0a502bd, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:27.350875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5a0f2-a941-450c-8386-f24425d7572f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:27.379255 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1446/2878] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:20:27 2022] PUT /identity/v3/projects/9121c63f41da43828687fd8e01a18963/users/dcc68c9d4c274f16946d8fb2e0a502bd/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:27.398488 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76eb874-2906-4e35-977c-d2b771b968a7 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:27.398899 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76eb874-2906-4e35-977c-d2b771b968a7 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:27.399330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76eb874-2906-4e35-977c-d2b771b968a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:27.448738 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-e76eb874-2906-4e35-977c-d2b771b968a7 None None] MFA Rules not processed for user `dcc68c9d4c274f16946d8fb2e0a502bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:20:27.450505 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-e76eb874-2906-4e35-977c-d2b771b968a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:27.517056 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:27.567680 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1433/2879] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:20:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:20:27.572727 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5a0f2-a941-450c-8386-f24425d7572f admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:27.645503 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5a0f2-a941-450c-8386-f24425d7572f admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:27.650432 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b81d1-abbc-49f6-8c1d-96d30b6b1334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:27.651096 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b81d1-abbc-49f6-8c1d-96d30b6b1334 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:27.651441 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b81d1-abbc-49f6-8c1d-96d30b6b1334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:27.651789 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b81d1-abbc-49f6-8c1d-96d30b6b1334 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:27.652707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b81d1-abbc-49f6-8c1d-96d30b6b1334 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:27.655092 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b81d1-abbc-49f6-8c1d-96d30b6b1334 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:27.687029 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1447/2880] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:20:27 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 05:20:27.692413 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76eb874-2906-4e35-977c-d2b771b968a7 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:27.705267 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76eb874-2906-4e35-977c-d2b771b968a7 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:27.773114 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cff709-760c-4208-97f8-5e7e83700425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:27.773931 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cff709-760c-4208-97f8-5e7e83700425 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:27.774299 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cff709-760c-4208-97f8-5e7e83700425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:27.774686 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cff709-760c-4208-97f8-5e7e83700425 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:27.775551 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cff709-760c-4208-97f8-5e7e83700425 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:27.777730 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cff709-760c-4208-97f8-5e7e83700425 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:27.802466 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1434/2881] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:20:27 2022] POST /identity/v3/users => generated 300 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 05:20:27.806956 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b81d1-abbc-49f6-8c1d-96d30b6b1334 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:27.818940 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b81d1-abbc-49f6-8c1d-96d30b6b1334 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:27.823921 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd8121-31e3-4423-851b-bf3e8bfd775c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:27.824543 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd8121-31e3-4423-851b-bf3e8bfd775c admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:27.824829 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd8121-31e3-4423-851b-bf3e8bfd775c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:27.825104 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd8121-31e3-4423-851b-bf3e8bfd775c admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:27.825838 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd8121-31e3-4423-851b-bf3e8bfd775c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:27.827991 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd8121-31e3-4423-851b-bf3e8bfd775c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:27.839862 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1448/2882] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 05:20:27.846619 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cff709-760c-4208-97f8-5e7e83700425 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:27.857535 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cff709-760c-4208-97f8-5e7e83700425 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:27.862634 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b43ee7-3cdf-4268-9619-4338c74ed17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:27.863306 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b43ee7-3cdf-4268-9619-4338c74ed17d admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:27.863590 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b43ee7-3cdf-4268-9619-4338c74ed17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:27.863866 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b43ee7-3cdf-4268-9619-4338c74ed17d admin admin] PATH_INFO: `/v3/projects/fc020d57bd60424c9dbd7dc3901f9933/users/d0712daf89db4198b71599a70343564c/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:27.890574 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b43ee7-3cdf-4268-9619-4338c74ed17d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc020d57bd60424c9dbd7dc3901f9933, user_id=d0712daf89db4198b71599a70343564c, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:27.895817 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b43ee7-3cdf-4268-9619-4338c74ed17d admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:27.919629 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1435/2883] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:20:27 2022] PUT /identity/v3/projects/fc020d57bd60424c9dbd7dc3901f9933/users/d0712daf89db4198b71599a70343564c/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:27.925709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd8121-31e3-4423-851b-bf3e8bfd775c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:27.942972 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd8121-31e3-4423-851b-bf3e8bfd775c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:28.011898 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42062e95-1fe0-474c-a3cd-a6c38c773d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:28.012831 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42062e95-1fe0-474c-a3cd-a6c38c773d6c admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:28.013308 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42062e95-1fe0-474c-a3cd-a6c38c773d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:28.013808 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42062e95-1fe0-474c-a3cd-a6c38c773d6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:28.015543 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42062e95-1fe0-474c-a3cd-a6c38c773d6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:28.018418 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42062e95-1fe0-474c-a3cd-a6c38c773d6c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:28.034657 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1449/2884] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 05:20:28.041235 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b43ee7-3cdf-4268-9619-4338c74ed17d admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:28.059588 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b43ee7-3cdf-4268-9619-4338c74ed17d admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:28.068158 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b82682-cac8-4f7c-a739-2d5fdc772b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:28.069233 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b82682-cac8-4f7c-a739-2d5fdc772b13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:28.069753 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b82682-cac8-4f7c-a739-2d5fdc772b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:28.070221 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b82682-cac8-4f7c-a739-2d5fdc772b13 admin admin] PATH_INFO: `/v3/projects/fc020d57bd60424c9dbd7dc3901f9933/users/d0712daf89db4198b71599a70343564c/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:28.073327 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b82682-cac8-4f7c-a739-2d5fdc772b13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc020d57bd60424c9dbd7dc3901f9933, user_id=d0712daf89db4198b71599a70343564c, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:28.076650 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b82682-cac8-4f7c-a739-2d5fdc772b13 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:28.089023 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1436/2885] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:20:28 2022] PUT /identity/v3/projects/fc020d57bd60424c9dbd7dc3901f9933/users/d0712daf89db4198b71599a70343564c/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:28.102806 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec867c7-2948-45cb-a4ae-7d452f722d58 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:28.103140 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec867c7-2948-45cb-a4ae-7d452f722d58 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:28.103455 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec867c7-2948-45cb-a4ae-7d452f722d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:28.135942 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-fec867c7-2948-45cb-a4ae-7d452f722d58 None None] MFA Rules not processed for user `d0712daf89db4198b71599a70343564c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:20:28.137232 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fec867c7-2948-45cb-a4ae-7d452f722d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:28.143815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:28.250528 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1450/2886] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:20:28 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:20:28.267697 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b82682-cac8-4f7c-a739-2d5fdc772b13 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:28.270250 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-49b82682-cac8-4f7c-a739-2d5fdc772b13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:28.356761 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b82682-cac8-4f7c-a739-2d5fdc772b13 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:28.361075 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f4e0b-e4d1-4a41-a967-c0d26e06e362 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:28.362557 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f4e0b-e4d1-4a41-a967-c0d26e06e362 service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:28.362926 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f4e0b-e4d1-4a41-a967-c0d26e06e362 service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:28.363819 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f4e0b-e4d1-4a41-a967-c0d26e06e362 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:28.377773 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f4e0b-e4d1-4a41-a967-c0d26e06e362 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:28.380476 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f4e0b-e4d1-4a41-a967-c0d26e06e362 service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:28.396898 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1437/2887] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:20:28 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 05:20:29.199855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec867c7-2948-45cb-a4ae-7d452f722d58 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:29.201524 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fec867c7-2948-45cb-a4ae-7d452f722d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:29.214985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec867c7-2948-45cb-a4ae-7d452f722d58 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:29.219492 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c9552-4e68-45c9-9d85-b84ac83827d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:29.220111 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c9552-4e68-45c9-9d85-b84ac83827d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:29.220398 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c9552-4e68-45c9-9d85-b84ac83827d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:29.220674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c9552-4e68-45c9-9d85-b84ac83827d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:29.293379 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c9552-4e68-45c9-9d85-b84ac83827d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:29.294800 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c9552-4e68-45c9-9d85-b84ac83827d4 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:29.304916 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1451/2888] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:20:29 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:20:33.823974 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f4e0b-e4d1-4a41-a967-c0d26e06e362 service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:33.841061 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f4e0b-e4d1-4a41-a967-c0d26e06e362 service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:33.848447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43501f8b-fd58-4a05-92f0-97ebb8a90398 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:33.849441 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43501f8b-fd58-4a05-92f0-97ebb8a90398 service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:33.850028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43501f8b-fd58-4a05-92f0-97ebb8a90398 service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:33.850480 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43501f8b-fd58-4a05-92f0-97ebb8a90398 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:33.907032 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:33.957331 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43501f8b-fd58-4a05-92f0-97ebb8a90398 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:33.960146 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43501f8b-fd58-4a05-92f0-97ebb8a90398 service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:33.982607 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1438/2889] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:20:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:20:35.359778 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c9552-4e68-45c9-9d85-b84ac83827d4 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:35.361643 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a44c9552-4e68-45c9-9d85-b84ac83827d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:35.445387 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c9552-4e68-45c9-9d85-b84ac83827d4 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:35.452094 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede3c60-3820-4668-9880-c7593fcc8e73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:35.453080 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede3c60-3820-4668-9880-c7593fcc8e73 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:35.453498 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede3c60-3820-4668-9880-c7593fcc8e73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:35.454024 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede3c60-3820-4668-9880-c7593fcc8e73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:35.456348 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7ede3c60-3820-4668-9880-c7593fcc8e73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:35.470331 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede3c60-3820-4668-9880-c7593fcc8e73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:35.471985 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede3c60-3820-4668-9880-c7593fcc8e73 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:35.482070 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1452/2890] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:20:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:20:36.851803 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43501f8b-fd58-4a05-92f0-97ebb8a90398 service cinder] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:36.950443 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43501f8b-fd58-4a05-92f0-97ebb8a90398 service cinder] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:36.956893 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249203e-09b2-4273-89dc-68cd03b2a430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:36.957876 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249203e-09b2-4273-89dc-68cd03b2a430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:36.958263 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249203e-09b2-4273-89dc-68cd03b2a430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:36.958583 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249203e-09b2-4273-89dc-68cd03b2a430 admin admin] PATH_INFO: `/v3/users/0e0c6e41740944d7ab5087cdfc444e70` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:36.960220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249203e-09b2-4273-89dc-68cd03b2a430 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e0c6e41740944d7ab5087cdfc444e70)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:36.963075 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249203e-09b2-4273-89dc-68cd03b2a430 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:37.035067 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-2249203e-09b2-4273-89dc-68cd03b2a430 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e0c6e41740944d7ab5087cdfc444e70'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:37.036192 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-2249203e-09b2-4273-89dc-68cd03b2a430 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e0c6e41740944d7ab5087cdfc444e70'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:37.058000 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-2249203e-09b2-4273-89dc-68cd03b2a430 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e0c6e41740944d7ab5087cdfc444e70'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:37.070633 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-2249203e-09b2-4273-89dc-68cd03b2a430 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e0c6e41740944d7ab5087cdfc444e70'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:37.071602 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-2249203e-09b2-4273-89dc-68cd03b2a430 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e0c6e41740944d7ab5087cdfc444e70'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:37.094937 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1439/2891] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:20:36 2022] DELETE /identity/v3/users/0e0c6e41740944d7ab5087cdfc444e70 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:37.102076 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e227f-fab0-430b-942c-2f902d9a29da None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:37.102642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e227f-fab0-430b-942c-2f902d9a29da None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:37.103007 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e227f-fab0-430b-942c-2f902d9a29da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:37.135900 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-c22e227f-fab0-430b-942c-2f902d9a29da None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:20:37.137951 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c22e227f-fab0-430b-942c-2f902d9a29da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:37.212994 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:37.249129 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1453/2892] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:20:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:20:37.263028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249203e-09b2-4273-89dc-68cd03b2a430 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:37.265301 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-2249203e-09b2-4273-89dc-68cd03b2a430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:37.350488 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249203e-09b2-4273-89dc-68cd03b2a430 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:37.356843 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747f3b9-c0a6-4dff-bd31-7d3df7639ea0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:37.358026 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747f3b9-c0a6-4dff-bd31-7d3df7639ea0 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:37.359052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747f3b9-c0a6-4dff-bd31-7d3df7639ea0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:37.359534 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747f3b9-c0a6-4dff-bd31-7d3df7639ea0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:37.370798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:37.378632 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747f3b9-c0a6-4dff-bd31-7d3df7639ea0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:37.381447 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747f3b9-c0a6-4dff-bd31-7d3df7639ea0 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:37.396948 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1440/2893] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:20:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 05:20:37.816761 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e227f-fab0-430b-942c-2f902d9a29da None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:37.819022 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c22e227f-fab0-430b-942c-2f902d9a29da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:37.847510 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e227f-fab0-430b-942c-2f902d9a29da None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:37.857765 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6fedc-b0c9-47de-b344-42a847a543f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:37.860785 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6fedc-b0c9-47de-b344-42a847a543f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:37.860785 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6fedc-b0c9-47de-b344-42a847a543f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:37.860785 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6fedc-b0c9-47de-b344-42a847a543f2 admin admin] PATH_INFO: `/v3/projects/2e50f256740c447a821e0ca76787cc3a` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:37.867549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6fedc-b0c9-47de-b344-42a847a543f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e50f256740c447a821e0ca76787cc3a)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:37.867549 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6fedc-b0c9-47de-b344-42a847a543f2 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:37.943478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-25d6fedc-b0c9-47de-b344-42a847a543f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e50f256740c447a821e0ca76787cc3a', 'request_id': 'req-25d6fedc-b0c9-47de-b344-42a847a543f2'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:37.955724 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-25d6fedc-b0c9-47de-b344-42a847a543f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e50f256740c447a821e0ca76787cc3a', 'request_id': 'req-25d6fedc-b0c9-47de-b344-42a847a543f2'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:37.970805 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-25d6fedc-b0c9-47de-b344-42a847a543f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e50f256740c447a821e0ca76787cc3a', 'request_id': 'req-25d6fedc-b0c9-47de-b344-42a847a543f2'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:37.992715 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-25d6fedc-b0c9-47de-b344-42a847a543f2 admin admin] The token cache is being invalidate because project 2e50f256740c447a821e0ca76787cc3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:20:37.993240 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-25d6fedc-b0c9-47de-b344-42a847a543f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:37.994304 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-25d6fedc-b0c9-47de-b344-42a847a543f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:37.996772 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1454/2894] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:20:37 2022] DELETE /identity/v3/projects/2e50f256740c447a821e0ca76787cc3a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:38.003450 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747f3b9-c0a6-4dff-bd31-7d3df7639ea0 service neutron] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:38.019277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747f3b9-c0a6-4dff-bd31-7d3df7639ea0 service neutron] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:38.114730 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ab1f2-9e66-4373-aa0a-daf972a2f780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:38.115378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ab1f2-9e66-4373-aa0a-daf972a2f780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:38.115720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ab1f2-9e66-4373-aa0a-daf972a2f780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:38.116034 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ab1f2-9e66-4373-aa0a-daf972a2f780 admin admin] PATH_INFO: `/v3/users/dcc68c9d4c274f16946d8fb2e0a502bd` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:38.117514 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ab1f2-9e66-4373-aa0a-daf972a2f780 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcc68c9d4c274f16946d8fb2e0a502bd)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:38.120136 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ab1f2-9e66-4373-aa0a-daf972a2f780 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:38.191471 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-984ab1f2-9e66-4373-aa0a-daf972a2f780 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcc68c9d4c274f16946d8fb2e0a502bd'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:38.192395 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-984ab1f2-9e66-4373-aa0a-daf972a2f780 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcc68c9d4c274f16946d8fb2e0a502bd'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:38.210752 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-984ab1f2-9e66-4373-aa0a-daf972a2f780 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcc68c9d4c274f16946d8fb2e0a502bd'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:38.224417 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-984ab1f2-9e66-4373-aa0a-daf972a2f780 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcc68c9d4c274f16946d8fb2e0a502bd'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:38.225376 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-984ab1f2-9e66-4373-aa0a-daf972a2f780 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcc68c9d4c274f16946d8fb2e0a502bd'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:38.250081 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1441/2895] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:20:38 2022] DELETE /identity/v3/users/dcc68c9d4c274f16946d8fb2e0a502bd => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:38.724929 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6fedc-b0c9-47de-b344-42a847a543f2 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:38.726893 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-25d6fedc-b0c9-47de-b344-42a847a543f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:38.741950 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6fedc-b0c9-47de-b344-42a847a543f2 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:38.836659 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78db4fe-64b2-44be-925f-d30ef405fa75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:38.837471 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78db4fe-64b2-44be-925f-d30ef405fa75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:38.837844 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78db4fe-64b2-44be-925f-d30ef405fa75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:38.838163 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78db4fe-64b2-44be-925f-d30ef405fa75 admin admin] PATH_INFO: `/v3/projects/9121c63f41da43828687fd8e01a18963` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:38.839559 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78db4fe-64b2-44be-925f-d30ef405fa75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9121c63f41da43828687fd8e01a18963)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:38.841648 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78db4fe-64b2-44be-925f-d30ef405fa75 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:38.917191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a78db4fe-64b2-44be-925f-d30ef405fa75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9121c63f41da43828687fd8e01a18963', 'request_id': 'req-a78db4fe-64b2-44be-925f-d30ef405fa75'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:38.928303 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a78db4fe-64b2-44be-925f-d30ef405fa75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9121c63f41da43828687fd8e01a18963', 'request_id': 'req-a78db4fe-64b2-44be-925f-d30ef405fa75'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:38.935344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a78db4fe-64b2-44be-925f-d30ef405fa75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9121c63f41da43828687fd8e01a18963', 'request_id': 'req-a78db4fe-64b2-44be-925f-d30ef405fa75'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:38.943855 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:38.950707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a78db4fe-64b2-44be-925f-d30ef405fa75 admin admin] The token cache is being invalidate because project 9121c63f41da43828687fd8e01a18963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:20:38.951459 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a78db4fe-64b2-44be-925f-d30ef405fa75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:38.953079 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a78db4fe-64b2-44be-925f-d30ef405fa75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:38.956000 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1455/2896] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:20:38 2022] DELETE /identity/v3/projects/9121c63f41da43828687fd8e01a18963 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:38.962602 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ab1f2-9e66-4373-aa0a-daf972a2f780 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:38.965913 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-984ab1f2-9e66-4373-aa0a-daf972a2f780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:38.991458 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ab1f2-9e66-4373-aa0a-daf972a2f780 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:39.085177 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936de454-0be5-475c-95e5-61c1b2bfb6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:39.086026 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936de454-0be5-475c-95e5-61c1b2bfb6f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:39.086426 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936de454-0be5-475c-95e5-61c1b2bfb6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:39.086761 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936de454-0be5-475c-95e5-61c1b2bfb6f4 admin admin] PATH_INFO: `/v3/users/d0712daf89db4198b71599a70343564c` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:39.088022 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936de454-0be5-475c-95e5-61c1b2bfb6f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0712daf89db4198b71599a70343564c)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:39.089985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936de454-0be5-475c-95e5-61c1b2bfb6f4 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:39.107384 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:39.161362 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-936de454-0be5-475c-95e5-61c1b2bfb6f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0712daf89db4198b71599a70343564c'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:39.162532 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-936de454-0be5-475c-95e5-61c1b2bfb6f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0712daf89db4198b71599a70343564c'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:39.185785 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-936de454-0be5-475c-95e5-61c1b2bfb6f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0712daf89db4198b71599a70343564c'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:39.196311 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-936de454-0be5-475c-95e5-61c1b2bfb6f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0712daf89db4198b71599a70343564c'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:39.197333 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-936de454-0be5-475c-95e5-61c1b2bfb6f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0712daf89db4198b71599a70343564c'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:39.218045 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1442/2897] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:20:38 2022] DELETE /identity/v3/users/d0712daf89db4198b71599a70343564c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:39.699318 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78db4fe-64b2-44be-925f-d30ef405fa75 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:39.701775 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a78db4fe-64b2-44be-925f-d30ef405fa75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:39.719006 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78db4fe-64b2-44be-925f-d30ef405fa75 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:39.791906 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5028b8de-2d30-45b2-a1b5-e1a0b6ade1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:39.792996 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028b8de-2d30-45b2-a1b5-e1a0b6ade1cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:39.793256 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028b8de-2d30-45b2-a1b5-e1a0b6ade1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:39.793779 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028b8de-2d30-45b2-a1b5-e1a0b6ade1cf admin admin] PATH_INFO: `/v3/projects/fc020d57bd60424c9dbd7dc3901f9933` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:39.796269 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5028b8de-2d30-45b2-a1b5-e1a0b6ade1cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc020d57bd60424c9dbd7dc3901f9933)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:39.798598 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5028b8de-2d30-45b2-a1b5-e1a0b6ade1cf admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:39.857697 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5028b8de-2d30-45b2-a1b5-e1a0b6ade1cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc020d57bd60424c9dbd7dc3901f9933', 'request_id': 'req-5028b8de-2d30-45b2-a1b5-e1a0b6ade1cf'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:39.868790 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5028b8de-2d30-45b2-a1b5-e1a0b6ade1cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc020d57bd60424c9dbd7dc3901f9933', 'request_id': 'req-5028b8de-2d30-45b2-a1b5-e1a0b6ade1cf'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:39.876385 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5028b8de-2d30-45b2-a1b5-e1a0b6ade1cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc020d57bd60424c9dbd7dc3901f9933', 'request_id': 'req-5028b8de-2d30-45b2-a1b5-e1a0b6ade1cf'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:39.892320 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5028b8de-2d30-45b2-a1b5-e1a0b6ade1cf admin admin] The token cache is being invalidate because project fc020d57bd60424c9dbd7dc3901f9933 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:20:39.892731 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5028b8de-2d30-45b2-a1b5-e1a0b6ade1cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:39.893705 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5028b8de-2d30-45b2-a1b5-e1a0b6ade1cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:39.896303 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1456/2898] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:20:39 2022] DELETE /identity/v3/projects/fc020d57bd60424c9dbd7dc3901f9933 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 05:20:39.906876 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02a75e-b1e4-4f94-aae1-476b858b7268 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:39.907351 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02a75e-b1e4-4f94-aae1-476b858b7268 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:39.908081 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02a75e-b1e4-4f94-aae1-476b858b7268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:39.942401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-9a02a75e-b1e4-4f94-aae1-476b858b7268 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:20:39.944313 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-9a02a75e-b1e4-4f94-aae1-476b858b7268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:40.029172 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1443/2899] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:20:40.042581 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f811cb5-1e30-4429-ac3d-b1e253fbc7d3 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:40.043073 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f811cb5-1e30-4429-ac3d-b1e253fbc7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:40.043552 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f811cb5-1e30-4429-ac3d-b1e253fbc7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:40.075959 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-4f811cb5-1e30-4429-ac3d-b1e253fbc7d3 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:20:40.079020 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4f811cb5-1e30-4429-ac3d-b1e253fbc7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:40.093045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1457/2900] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:20:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:20:40.099477 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02a75e-b1e4-4f94-aae1-476b858b7268 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:40.116949 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02a75e-b1e4-4f94-aae1-476b858b7268 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:40.125194 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e45e5-4475-4933-a2d6-04d7fe7620fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:40.126209 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e45e5-4475-4933-a2d6-04d7fe7620fe admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:40.126699 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e45e5-4475-4933-a2d6-04d7fe7620fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:40.127097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e45e5-4475-4933-a2d6-04d7fe7620fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:40.128298 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e45e5-4475-4933-a2d6-04d7fe7620fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 05:20:40.128807 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e45e5-4475-4933-a2d6-04d7fe7620fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:40.131782 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e45e5-4475-4933-a2d6-04d7fe7620fe admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:40.153485 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1444/2901] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 05:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 05:20:40.159344 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f811cb5-1e30-4429-ac3d-b1e253fbc7d3 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:40.169386 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f811cb5-1e30-4429-ac3d-b1e253fbc7d3 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:40.174844 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867b733-7073-4be8-8053-f4f7d58f521f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:40.175470 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867b733-7073-4be8-8053-f4f7d58f521f admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:40.175786 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867b733-7073-4be8-8053-f4f7d58f521f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:40.176072 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867b733-7073-4be8-8053-f4f7d58f521f admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:40.176892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2867b733-7073-4be8-8053-f4f7d58f521f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:40.178954 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2867b733-7073-4be8-8053-f4f7d58f521f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:40.212709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1458/2902] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:20:40 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 05:20:40.218294 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e45e5-4475-4933-a2d6-04d7fe7620fe admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:40.233891 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e45e5-4475-4933-a2d6-04d7fe7620fe admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:40.320533 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78a208-1752-4ea4-b7ab-47f1b870457c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:40.321890 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78a208-1752-4ea4-b7ab-47f1b870457c admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:40.322357 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78a208-1752-4ea4-b7ab-47f1b870457c admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:40.322820 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78a208-1752-4ea4-b7ab-47f1b870457c admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:40.323981 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78a208-1752-4ea4-b7ab-47f1b870457c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:40.327308 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78a208-1752-4ea4-b7ab-47f1b870457c admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:40.353961 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1445/2903] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:20:40 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:20:40.360064 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867b733-7073-4be8-8053-f4f7d58f521f admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:40.371143 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867b733-7073-4be8-8053-f4f7d58f521f admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:40.377940 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b4eec3-803f-408d-a020-e99206a466e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:40.378650 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4eec3-803f-408d-a020-e99206a466e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:40.378931 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4eec3-803f-408d-a020-e99206a466e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:40.379203 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4eec3-803f-408d-a020-e99206a466e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:40.379926 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b4eec3-803f-408d-a020-e99206a466e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:40.381532 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b4eec3-803f-408d-a020-e99206a466e1 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:40.393984 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1459/2904] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:20:40.399484 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78a208-1752-4ea4-b7ab-47f1b870457c admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:40.411264 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78a208-1752-4ea4-b7ab-47f1b870457c admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:40.416749 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8325a-5c5e-47f9-94e9-db8bcba10aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:40.417374 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8325a-5c5e-47f9-94e9-db8bcba10aff admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:40.417700 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8325a-5c5e-47f9-94e9-db8bcba10aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:40.417999 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8325a-5c5e-47f9-94e9-db8bcba10aff admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:40.418829 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8325a-5c5e-47f9-94e9-db8bcba10aff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:40.421576 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8325a-5c5e-47f9-94e9-db8bcba10aff admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:40.424888 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:40.442065 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1446/2905] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:20:40.446649 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b4eec3-803f-408d-a020-e99206a466e1 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:40.458908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b4eec3-803f-408d-a020-e99206a466e1 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:40.464475 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693d215-4a30-4d3c-9eb1-868d619525aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:40.465296 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693d215-4a30-4d3c-9eb1-868d619525aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:40.465693 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693d215-4a30-4d3c-9eb1-868d619525aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:40.466110 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693d215-4a30-4d3c-9eb1-868d619525aa admin admin] PATH_INFO: `/v3/projects/51f83daef6d5406daa539f63d4e23f15/users/f5aee3c64e3f48e995193fcf7bba7bb4/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:40.494892 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6693d215-4a30-4d3c-9eb1-868d619525aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=51f83daef6d5406daa539f63d4e23f15, user_id=f5aee3c64e3f48e995193fcf7bba7bb4, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:40.497907 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6693d215-4a30-4d3c-9eb1-868d619525aa admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:40.516668 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1460/2906] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:20:40 2022] PUT /identity/v3/projects/51f83daef6d5406daa539f63d4e23f15/users/f5aee3c64e3f48e995193fcf7bba7bb4/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:40.528359 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7deb0-a0d2-4b67-9205-01344320b901 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:40.528647 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7deb0-a0d2-4b67-9205-01344320b901 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:40.528925 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7deb0-a0d2-4b67-9205-01344320b901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:40.564824 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-94a7deb0-a0d2-4b67-9205-01344320b901 None None] MFA Rules not processed for user `f5aee3c64e3f48e995193fcf7bba7bb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:20:40.566704 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-94a7deb0-a0d2-4b67-9205-01344320b901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:40.683862 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1447/2907] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:20:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 05:20:40.704570 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693d215-4a30-4d3c-9eb1-868d619525aa admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:40.706614 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6693d215-4a30-4d3c-9eb1-868d619525aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:40.740322 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:40.782730 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693d215-4a30-4d3c-9eb1-868d619525aa admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:40.788472 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9e460-bfb6-42c1-85f4-72531aceb15f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:40.789517 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9e460-bfb6-42c1-85f4-72531aceb15f service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:40.790053 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9e460-bfb6-42c1-85f4-72531aceb15f service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:40.790541 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9e460-bfb6-42c1-85f4-72531aceb15f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:40.805013 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9e460-bfb6-42c1-85f4-72531aceb15f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:40.806628 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9e460-bfb6-42c1-85f4-72531aceb15f service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:40.823124 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1461/2908] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:20:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:20:49.951966 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7deb0-a0d2-4b67-9205-01344320b901 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:50.045825 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7deb0-a0d2-4b67-9205-01344320b901 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:50.050717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06544dd5-09e0-4919-8b08-407e9498c4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:50.051801 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06544dd5-09e0-4919-8b08-407e9498c4f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:50.051967 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06544dd5-09e0-4919-8b08-407e9498c4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:50.052303 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06544dd5-09e0-4919-8b08-407e9498c4f0 admin admin] PATH_INFO: `/v3/users/f5aee3c64e3f48e995193fcf7bba7bb4` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:50.053619 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06544dd5-09e0-4919-8b08-407e9498c4f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5aee3c64e3f48e995193fcf7bba7bb4)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:50.055624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06544dd5-09e0-4919-8b08-407e9498c4f0 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:50.115341 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-06544dd5-09e0-4919-8b08-407e9498c4f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5aee3c64e3f48e995193fcf7bba7bb4'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:50.116237 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-06544dd5-09e0-4919-8b08-407e9498c4f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5aee3c64e3f48e995193fcf7bba7bb4'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:50.133733 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-06544dd5-09e0-4919-8b08-407e9498c4f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5aee3c64e3f48e995193fcf7bba7bb4'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:50.142130 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-06544dd5-09e0-4919-8b08-407e9498c4f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5aee3c64e3f48e995193fcf7bba7bb4'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:50.142891 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-06544dd5-09e0-4919-8b08-407e9498c4f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5aee3c64e3f48e995193fcf7bba7bb4'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:50.157794 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1448/2909] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:20:49 2022] DELETE /identity/v3/users/f5aee3c64e3f48e995193fcf7bba7bb4 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:50.163062 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74977840-24ea-44e8-b77a-c31ee7e318ca None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:50.163637 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74977840-24ea-44e8-b77a-c31ee7e318ca None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:50.163767 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74977840-24ea-44e8-b77a-c31ee7e318ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:50.197497 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-74977840-24ea-44e8-b77a-c31ee7e318ca None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:20:50.198655 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-74977840-24ea-44e8-b77a-c31ee7e318ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:50.267642 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1462/2910] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:20:50.280220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06544dd5-09e0-4919-8b08-407e9498c4f0 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:50.281746 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-06544dd5-09e0-4919-8b08-407e9498c4f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:50.293499 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:50.365850 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06544dd5-09e0-4919-8b08-407e9498c4f0 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:50.370672 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e36e7c-91d3-4263-afff-2c9974744e54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:50.371315 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e36e7c-91d3-4263-afff-2c9974744e54 service neutron] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:50.371645 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e36e7c-91d3-4263-afff-2c9974744e54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:50.372149 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e36e7c-91d3-4263-afff-2c9974744e54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:50.389845 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e36e7c-91d3-4263-afff-2c9974744e54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:50.392509 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e36e7c-91d3-4263-afff-2c9974744e54 service neutron] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:50.407165 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1449/2911] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:20:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 05:20:50.789928 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74977840-24ea-44e8-b77a-c31ee7e318ca None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:50.792402 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-74977840-24ea-44e8-b77a-c31ee7e318ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:50.814375 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74977840-24ea-44e8-b77a-c31ee7e318ca None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:50.824407 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60a2c8-70e0-4b2d-b531-16d245a945be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:50.825493 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60a2c8-70e0-4b2d-b531-16d245a945be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:50.826707 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60a2c8-70e0-4b2d-b531-16d245a945be admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:50.827207 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60a2c8-70e0-4b2d-b531-16d245a945be admin admin] PATH_INFO: `/v3/projects/51f83daef6d5406daa539f63d4e23f15` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:50.829380 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60a2c8-70e0-4b2d-b531-16d245a945be admin admin] RBAC: Authorizing `identity:delete_project(project_id=51f83daef6d5406daa539f63d4e23f15)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:50.832635 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60a2c8-70e0-4b2d-b531-16d245a945be admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:50.901567 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-cd60a2c8-70e0-4b2d-b531-16d245a945be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51f83daef6d5406daa539f63d4e23f15', 'request_id': 'req-cd60a2c8-70e0-4b2d-b531-16d245a945be'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:50.911579 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-cd60a2c8-70e0-4b2d-b531-16d245a945be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51f83daef6d5406daa539f63d4e23f15', 'request_id': 'req-cd60a2c8-70e0-4b2d-b531-16d245a945be'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:50.917603 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-cd60a2c8-70e0-4b2d-b531-16d245a945be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51f83daef6d5406daa539f63d4e23f15', 'request_id': 'req-cd60a2c8-70e0-4b2d-b531-16d245a945be'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:50.930261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-cd60a2c8-70e0-4b2d-b531-16d245a945be admin admin] The token cache is being invalidate because project 51f83daef6d5406daa539f63d4e23f15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:20:50.930592 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-cd60a2c8-70e0-4b2d-b531-16d245a945be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:50.931383 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-cd60a2c8-70e0-4b2d-b531-16d245a945be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:50.933067 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1463/2912] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:20:50 2022] DELETE /identity/v3/projects/51f83daef6d5406daa539f63d4e23f15 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:50.939049 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6155e-b1cb-48a1-b4f9-ba38fb19a8e3 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:50.939462 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6155e-b1cb-48a1-b4f9-ba38fb19a8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:50.939712 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6155e-b1cb-48a1-b4f9-ba38fb19a8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:50.965499 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-cbd6155e-b1cb-48a1-b4f9-ba38fb19a8e3 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:20:50.968446 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-cbd6155e-b1cb-48a1-b4f9-ba38fb19a8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:51.050378 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1450/2913] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 05:20:51.062331 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9caa705-e470-40b2-aa5a-2483aba4f1bf None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:51.062655 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9caa705-e470-40b2-aa5a-2483aba4f1bf None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:51.062875 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9caa705-e470-40b2-aa5a-2483aba4f1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:51.092971 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-d9caa705-e470-40b2-aa5a-2483aba4f1bf None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:20:51.095093 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d9caa705-e470-40b2-aa5a-2483aba4f1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:51.109332 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1464/2914] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:20:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:20:51.114319 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd6155e-b1cb-48a1-b4f9-ba38fb19a8e3 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:51.130422 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd6155e-b1cb-48a1-b4f9-ba38fb19a8e3 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:51.138877 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cba2a1-b645-46dc-a319-19a78ac59111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:51.139947 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cba2a1-b645-46dc-a319-19a78ac59111 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:51.140380 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cba2a1-b645-46dc-a319-19a78ac59111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:51.140832 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cba2a1-b645-46dc-a319-19a78ac59111 admin admin] PATH_INFO: `/v3/domains` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:51.142055 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cba2a1-b645-46dc-a319-19a78ac59111 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 05:20:51.142610 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cba2a1-b645-46dc-a319-19a78ac59111 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:51.145637 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cba2a1-b645-46dc-a319-19a78ac59111 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:51.164175 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1451/2915] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 05:20:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 05:20:51.168754 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9caa705-e470-40b2-aa5a-2483aba4f1bf None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:51.183012 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9caa705-e470-40b2-aa5a-2483aba4f1bf None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:51.188864 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78746a-a152-42b1-91fa-b793d590c821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:51.189580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78746a-a152-42b1-91fa-b793d590c821 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:51.189857 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78746a-a152-42b1-91fa-b793d590c821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:51.190126 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78746a-a152-42b1-91fa-b793d590c821 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:51.190803 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78746a-a152-42b1-91fa-b793d590c821 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:51.192736 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78746a-a152-42b1-91fa-b793d590c821 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:51.237787 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1465/2916] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:20:51 2022] POST /identity/v3/projects => generated 375 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:20:51.242253 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cba2a1-b645-46dc-a319-19a78ac59111 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:51.253745 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cba2a1-b645-46dc-a319-19a78ac59111 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:51.334427 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7dad06-4dfe-44a8-a18f-8815a6305366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:51.334976 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7dad06-4dfe-44a8-a18f-8815a6305366 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:51.335265 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7dad06-4dfe-44a8-a18f-8815a6305366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:51.335541 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7dad06-4dfe-44a8-a18f-8815a6305366 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:51.336232 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7dad06-4dfe-44a8-a18f-8815a6305366 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:51.338220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7dad06-4dfe-44a8-a18f-8815a6305366 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:51.365515 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1452/2917] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:20:51 2022] POST /identity/v3/users => generated 288 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 05:20:51.370515 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78746a-a152-42b1-91fa-b793d590c821 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:51.383181 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78746a-a152-42b1-91fa-b793d590c821 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:51.390008 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf6c37-0f69-47e6-bd33-434be4e9e985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:51.390933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf6c37-0f69-47e6-bd33-434be4e9e985 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:51.390933 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf6c37-0f69-47e6-bd33-434be4e9e985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:51.391231 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf6c37-0f69-47e6-bd33-434be4e9e985 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:51.391884 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cf6c37-0f69-47e6-bd33-434be4e9e985 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:51.393943 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cf6c37-0f69-47e6-bd33-434be4e9e985 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:51.409739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1466/2918] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:20:51.414733 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7dad06-4dfe-44a8-a18f-8815a6305366 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:51.427419 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7dad06-4dfe-44a8-a18f-8815a6305366 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:51.435101 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577372c-4e15-4e58-b909-6eec25226f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:51.436117 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577372c-4e15-4e58-b909-6eec25226f27 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:51.436616 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577372c-4e15-4e58-b909-6eec25226f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:51.437097 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577372c-4e15-4e58-b909-6eec25226f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:51.438364 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2577372c-4e15-4e58-b909-6eec25226f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:51.440521 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2577372c-4e15-4e58-b909-6eec25226f27 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:51.460368 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1453/2919] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:20:51.466237 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf6c37-0f69-47e6-bd33-434be4e9e985 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:51.485870 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf6c37-0f69-47e6-bd33-434be4e9e985 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:51.493871 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42ab21-db8f-4604-9a9e-0a129be4bd66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:51.495294 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42ab21-db8f-4604-9a9e-0a129be4bd66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:51.495796 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42ab21-db8f-4604-9a9e-0a129be4bd66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:51.496291 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42ab21-db8f-4604-9a9e-0a129be4bd66 admin admin] PATH_INFO: `/v3/projects/0cd6fab1e383455080228c5a6a0f806d/users/78291a414c7742efb46fcf1be15b2fb6/roles/e733372bae6047be956ad703f83a480e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:51.508580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:51.522220 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c42ab21-db8f-4604-9a9e-0a129be4bd66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cd6fab1e383455080228c5a6a0f806d, user_id=78291a414c7742efb46fcf1be15b2fb6, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:51.525796 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c42ab21-db8f-4604-9a9e-0a129be4bd66 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:51.549191 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1467/2920] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:20:51 2022] PUT /identity/v3/projects/0cd6fab1e383455080228c5a6a0f806d/users/78291a414c7742efb46fcf1be15b2fb6/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:51.570036 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f847b0a-0487-482c-ac7b-a63e50454ef0 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:51.570466 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f847b0a-0487-482c-ac7b-a63e50454ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:51.570986 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f847b0a-0487-482c-ac7b-a63e50454ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:51.613407 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-2f847b0a-0487-482c-ac7b-a63e50454ef0 None None] MFA Rules not processed for user `78291a414c7742efb46fcf1be15b2fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:20:51.614590 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-2f847b0a-0487-482c-ac7b-a63e50454ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:51.619018 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:51.691718 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1454/2921] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:20:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:20:51.696836 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42ab21-db8f-4604-9a9e-0a129be4bd66 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:51.769474 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42ab21-db8f-4604-9a9e-0a129be4bd66 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:51.775176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71b18f-324a-4ad2-ab8c-4bcae4e8c45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:51.775927 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71b18f-324a-4ad2-ab8c-4bcae4e8c45e admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:51.776321 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71b18f-324a-4ad2-ab8c-4bcae4e8c45e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:51.776666 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71b18f-324a-4ad2-ab8c-4bcae4e8c45e admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:51.777457 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71b18f-324a-4ad2-ab8c-4bcae4e8c45e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:51.780264 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71b18f-324a-4ad2-ab8c-4bcae4e8c45e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:51.812789 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1468/2922] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:20:51 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 05:20:51.819557 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f847b0a-0487-482c-ac7b-a63e50454ef0 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:51.836504 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f847b0a-0487-482c-ac7b-a63e50454ef0 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:51.914285 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04794116-ee33-4f4b-a248-4450d910c536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:51.915188 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04794116-ee33-4f4b-a248-4450d910c536 admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:51.915503 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04794116-ee33-4f4b-a248-4450d910c536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:51.915779 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04794116-ee33-4f4b-a248-4450d910c536 admin admin] PATH_INFO: `/v3/users` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:51.916779 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04794116-ee33-4f4b-a248-4450d910c536 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:51.920578 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04794116-ee33-4f4b-a248-4450d910c536 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:51.952367 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1455/2923] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:20:51 2022] POST /identity/v3/users => generated 294 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:20:51.957388 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71b18f-324a-4ad2-ab8c-4bcae4e8c45e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:51.970483 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71b18f-324a-4ad2-ab8c-4bcae4e8c45e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:51.975577 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60853a2a-0725-41b3-9fd3-219ca0b1a4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:51.976207 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60853a2a-0725-41b3-9fd3-219ca0b1a4ac admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:51.976492 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60853a2a-0725-41b3-9fd3-219ca0b1a4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:51.976775 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60853a2a-0725-41b3-9fd3-219ca0b1a4ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:51.977491 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60853a2a-0725-41b3-9fd3-219ca0b1a4ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:51.979123 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60853a2a-0725-41b3-9fd3-219ca0b1a4ac admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:51.991478 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1469/2924] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:20:51.998465 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04794116-ee33-4f4b-a248-4450d910c536 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:52.009324 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04794116-ee33-4f4b-a248-4450d910c536 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:52.015450 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98e1e0-951e-479e-a6e9-d6a29db43873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:52.016162 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98e1e0-951e-479e-a6e9-d6a29db43873 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:52.016443 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98e1e0-951e-479e-a6e9-d6a29db43873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:52.016720 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98e1e0-951e-479e-a6e9-d6a29db43873 admin admin] PATH_INFO: `/v3/projects/71b3964d53a54f0192f6310d21d5fb59/users/3ce5d16a070443ee8fd3b626831f265e/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:52.037598 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e98e1e0-951e-479e-a6e9-d6a29db43873 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b3964d53a54f0192f6310d21d5fb59, user_id=3ce5d16a070443ee8fd3b626831f265e, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:52.040525 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e98e1e0-951e-479e-a6e9-d6a29db43873 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:52.064448 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1456/2925] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:20:51 2022] PUT /identity/v3/projects/71b3964d53a54f0192f6310d21d5fb59/users/3ce5d16a070443ee8fd3b626831f265e/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 05:20:52.068785 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60853a2a-0725-41b3-9fd3-219ca0b1a4ac admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:52.086826 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60853a2a-0725-41b3-9fd3-219ca0b1a4ac admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:52.166621 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c0ceb-3048-4b9d-8f84-404ef3e47409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:52.167145 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c0ceb-3048-4b9d-8f84-404ef3e47409 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:52.167426 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c0ceb-3048-4b9d-8f84-404ef3e47409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:52.167701 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c0ceb-3048-4b9d-8f84-404ef3e47409 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:52.168411 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c0ceb-3048-4b9d-8f84-404ef3e47409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:52.170125 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c0ceb-3048-4b9d-8f84-404ef3e47409 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:52.181336 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1470/2926] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:20:52.186404 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98e1e0-951e-479e-a6e9-d6a29db43873 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:52.207885 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98e1e0-951e-479e-a6e9-d6a29db43873 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:52.216105 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c980b-a20d-478a-b1fd-e3f15c26902c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:52.218111 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c980b-a20d-478a-b1fd-e3f15c26902c admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:52.218349 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c980b-a20d-478a-b1fd-e3f15c26902c admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:52.219277 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c980b-a20d-478a-b1fd-e3f15c26902c admin admin] PATH_INFO: `/v3/projects/71b3964d53a54f0192f6310d21d5fb59/users/3ce5d16a070443ee8fd3b626831f265e/roles/4a6517e5890a49718303007dac0756c7` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:52.226316 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c980b-a20d-478a-b1fd-e3f15c26902c admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b3964d53a54f0192f6310d21d5fb59, user_id=3ce5d16a070443ee8fd3b626831f265e, role_id=4a6517e5890a49718303007dac0756c7)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:52.233846 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c980b-a20d-478a-b1fd-e3f15c26902c admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:52.258138 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1457/2927] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:20:52 2022] PUT /identity/v3/projects/71b3964d53a54f0192f6310d21d5fb59/users/3ce5d16a070443ee8fd3b626831f265e/roles/4a6517e5890a49718303007dac0756c7 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:52.276709 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45976c22-4017-49ee-92c7-3004b2abeb24 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:52.277242 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45976c22-4017-49ee-92c7-3004b2abeb24 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:52.277739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45976c22-4017-49ee-92c7-3004b2abeb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:52.327600 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-45976c22-4017-49ee-92c7-3004b2abeb24 None None] MFA Rules not processed for user `3ce5d16a070443ee8fd3b626831f265e`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:20:52.329128 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-45976c22-4017-49ee-92c7-3004b2abeb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:52.396250 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:52.442024 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1471/2928] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:20:52 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:20:52.459190 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c980b-a20d-478a-b1fd-e3f15c26902c admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:52.461819 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-f30c980b-a20d-478a-b1fd-e3f15c26902c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:52.553356 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c980b-a20d-478a-b1fd-e3f15c26902c admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:52.559616 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b8408-3862-4cab-936c-87edbdcf91ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:52.561205 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b8408-3862-4cab-936c-87edbdcf91ce service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:52.561719 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b8408-3862-4cab-936c-87edbdcf91ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:52.562164 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b8408-3862-4cab-936c-87edbdcf91ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:52.579433 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:52.656496 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b8408-3862-4cab-936c-87edbdcf91ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:52.658412 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b8408-3862-4cab-936c-87edbdcf91ce service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:52.672093 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1458/2929] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:20:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:20:55.433732 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45976c22-4017-49ee-92c7-3004b2abeb24 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:55.533081 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45976c22-4017-49ee-92c7-3004b2abeb24 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:55.539492 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e84a6-f008-4dc6-9378-eca286ba465b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:55.540435 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e84a6-f008-4dc6-9378-eca286ba465b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:55.540938 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e84a6-f008-4dc6-9378-eca286ba465b admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:55.541419 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e84a6-f008-4dc6-9378-eca286ba465b admin admin] PATH_INFO: `/v3/users/78291a414c7742efb46fcf1be15b2fb6` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:55.543530 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872e84a6-f008-4dc6-9378-eca286ba465b admin admin] RBAC: Authorizing `identity:delete_user(user_id=78291a414c7742efb46fcf1be15b2fb6)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:55.547317 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872e84a6-f008-4dc6-9378-eca286ba465b admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:55.624416 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-872e84a6-f008-4dc6-9378-eca286ba465b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78291a414c7742efb46fcf1be15b2fb6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:55.631974 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-872e84a6-f008-4dc6-9378-eca286ba465b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78291a414c7742efb46fcf1be15b2fb6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:55.632700 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-872e84a6-f008-4dc6-9378-eca286ba465b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78291a414c7742efb46fcf1be15b2fb6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:55.646300 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-872e84a6-f008-4dc6-9378-eca286ba465b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78291a414c7742efb46fcf1be15b2fb6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:55.647077 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-872e84a6-f008-4dc6-9378-eca286ba465b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78291a414c7742efb46fcf1be15b2fb6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:55.663448 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1472/2930] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:20:55 2022] DELETE /identity/v3/users/78291a414c7742efb46fcf1be15b2fb6 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:55.668915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a70c7d-d9b1-46b4-bcca-8870b949286e None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:55.670089 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a70c7d-d9b1-46b4-bcca-8870b949286e None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:55.670690 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a70c7d-d9b1-46b4-bcca-8870b949286e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:55.702254 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-05a70c7d-d9b1-46b4-bcca-8870b949286e None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:20:55.703557 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-05a70c7d-d9b1-46b4-bcca-8870b949286e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:55.785230 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1459/2931] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:20:55.798477 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e84a6-f008-4dc6-9378-eca286ba465b admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:55.803038 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-872e84a6-f008-4dc6-9378-eca286ba465b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:55.887724 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e84a6-f008-4dc6-9378-eca286ba465b admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:55.894328 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577afb62-a5da-40d5-8125-1505a9adb827 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:55.895373 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577afb62-a5da-40d5-8125-1505a9adb827 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:55.895835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577afb62-a5da-40d5-8125-1505a9adb827 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:55.896239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577afb62-a5da-40d5-8125-1505a9adb827 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:55.913406 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577afb62-a5da-40d5-8125-1505a9adb827 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:55.916371 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577afb62-a5da-40d5-8125-1505a9adb827 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:55.952631 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1473/2932] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:20:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:20:56.355999 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a70c7d-d9b1-46b4-bcca-8870b949286e None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:56.359376 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-05a70c7d-d9b1-46b4-bcca-8870b949286e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:56.388205 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a70c7d-d9b1-46b4-bcca-8870b949286e None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:56.395982 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17cd46-fa39-47bd-a7e0-2969d779a5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:56.396991 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17cd46-fa39-47bd-a7e0-2969d779a5b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:56.397474 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17cd46-fa39-47bd-a7e0-2969d779a5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:56.397929 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17cd46-fa39-47bd-a7e0-2969d779a5b4 admin admin] PATH_INFO: `/v3/projects/0cd6fab1e383455080228c5a6a0f806d` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:56.399956 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17cd46-fa39-47bd-a7e0-2969d779a5b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cd6fab1e383455080228c5a6a0f806d)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:56.403356 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17cd46-fa39-47bd-a7e0-2969d779a5b4 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:56.495345 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5b17cd46-fa39-47bd-a7e0-2969d779a5b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cd6fab1e383455080228c5a6a0f806d', 'request_id': 'req-5b17cd46-fa39-47bd-a7e0-2969d779a5b4'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:56.506354 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5b17cd46-fa39-47bd-a7e0-2969d779a5b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cd6fab1e383455080228c5a6a0f806d', 'request_id': 'req-5b17cd46-fa39-47bd-a7e0-2969d779a5b4'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:56.517224 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5b17cd46-fa39-47bd-a7e0-2969d779a5b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cd6fab1e383455080228c5a6a0f806d', 'request_id': 'req-5b17cd46-fa39-47bd-a7e0-2969d779a5b4'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:56.529935 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5b17cd46-fa39-47bd-a7e0-2969d779a5b4 admin admin] The token cache is being invalidate because project 0cd6fab1e383455080228c5a6a0f806d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:20:56.530255 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5b17cd46-fa39-47bd-a7e0-2969d779a5b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:56.530951 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-5b17cd46-fa39-47bd-a7e0-2969d779a5b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:56.532798 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1460/2933] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:20:56 2022] DELETE /identity/v3/projects/0cd6fab1e383455080228c5a6a0f806d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:56.538766 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577afb62-a5da-40d5-8125-1505a9adb827 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:56.545903 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:56.550092 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577afb62-a5da-40d5-8125-1505a9adb827 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:56.616169 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014923c-cb7d-4e97-968d-b2717753aca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:56.616819 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014923c-cb7d-4e97-968d-b2717753aca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:56.617093 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014923c-cb7d-4e97-968d-b2717753aca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:56.617367 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014923c-cb7d-4e97-968d-b2717753aca0 admin admin] PATH_INFO: `/v3/users/3ce5d16a070443ee8fd3b626831f265e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:56.618604 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014923c-cb7d-4e97-968d-b2717753aca0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ce5d16a070443ee8fd3b626831f265e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:56.620521 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014923c-cb7d-4e97-968d-b2717753aca0 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:56.686487 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a014923c-cb7d-4e97-968d-b2717753aca0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ce5d16a070443ee8fd3b626831f265e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:56.694531 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a014923c-cb7d-4e97-968d-b2717753aca0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ce5d16a070443ee8fd3b626831f265e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:56.695269 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a014923c-cb7d-4e97-968d-b2717753aca0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ce5d16a070443ee8fd3b626831f265e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:56.710574 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a014923c-cb7d-4e97-968d-b2717753aca0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ce5d16a070443ee8fd3b626831f265e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:56.711357 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a014923c-cb7d-4e97-968d-b2717753aca0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ce5d16a070443ee8fd3b626831f265e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:56.732739 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1474/2934] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:20:56 2022] DELETE /identity/v3/users/3ce5d16a070443ee8fd3b626831f265e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 05:20:57.130932 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17cd46-fa39-47bd-a7e0-2969d779a5b4 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:57.132975 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-5b17cd46-fa39-47bd-a7e0-2969d779a5b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:57.156424 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17cd46-fa39-47bd-a7e0-2969d779a5b4 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:57.214544 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:57.250444 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05595347-e4ea-4c83-a961-ced48c521b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:57.251564 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05595347-e4ea-4c83-a961-ced48c521b13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:57.251857 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05595347-e4ea-4c83-a961-ced48c521b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:57.252147 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05595347-e4ea-4c83-a961-ced48c521b13 admin admin] PATH_INFO: `/v3/projects/71b3964d53a54f0192f6310d21d5fb59` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:57.253470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05595347-e4ea-4c83-a961-ced48c521b13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71b3964d53a54f0192f6310d21d5fb59)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:57.256338 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05595347-e4ea-4c83-a961-ced48c521b13 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:57.343868 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-05595347-e4ea-4c83-a961-ced48c521b13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71b3964d53a54f0192f6310d21d5fb59', 'request_id': 'req-05595347-e4ea-4c83-a961-ced48c521b13'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:57.354656 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-05595347-e4ea-4c83-a961-ced48c521b13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71b3964d53a54f0192f6310d21d5fb59', 'request_id': 'req-05595347-e4ea-4c83-a961-ced48c521b13'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:57.362035 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-05595347-e4ea-4c83-a961-ced48c521b13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71b3964d53a54f0192f6310d21d5fb59', 'request_id': 'req-05595347-e4ea-4c83-a961-ced48c521b13'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:57.375734 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-05595347-e4ea-4c83-a961-ced48c521b13 admin admin] The token cache is being invalidate because project 71b3964d53a54f0192f6310d21d5fb59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:20:57.376094 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-05595347-e4ea-4c83-a961-ced48c521b13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:57.376814 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-05595347-e4ea-4c83-a961-ced48c521b13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:57.378780 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1461/2935] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:20:57 2022] DELETE /identity/v3/projects/71b3964d53a54f0192f6310d21d5fb59 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:57.385158 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba006af-ea80-4494-97bb-b8cc3e3f7e8e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:57.385682 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba006af-ea80-4494-97bb-b8cc3e3f7e8e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:57.386097 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba006af-ea80-4494-97bb-b8cc3e3f7e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:57.410195 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-8ba006af-ea80-4494-97bb-b8cc3e3f7e8e None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:20:57.411553 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8ba006af-ea80-4494-97bb-b8cc3e3f7e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:57.499449 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1475/2936] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:20:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:20:57.507814 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbff39-cad0-4227-ae1c-3955226f6d22 None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:57.508094 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbff39-cad0-4227-ae1c-3955226f6d22 None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:57.508381 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbff39-cad0-4227-ae1c-3955226f6d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:57.530525 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-c9fbff39-cad0-4227-ae1c-3955226f6d22 None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:20:57.531778 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-c9fbff39-cad0-4227-ae1c-3955226f6d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:57.543239 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1462/2937] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:20:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 05:20:57.550046 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba006af-ea80-4494-97bb-b8cc3e3f7e8e None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:57.568378 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba006af-ea80-4494-97bb-b8cc3e3f7e8e None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:57.573449 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ae98a-13c9-448a-ab20-b6e23232981c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:57.574104 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ae98a-13c9-448a-ab20-b6e23232981c admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:57.574381 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ae98a-13c9-448a-ab20-b6e23232981c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:57.574674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ae98a-13c9-448a-ab20-b6e23232981c admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:57.576324 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ae98a-13c9-448a-ab20-b6e23232981c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 05:20:57.576634 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ae98a-13c9-448a-ab20-b6e23232981c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:57.578610 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ae98a-13c9-448a-ab20-b6e23232981c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:57.592197 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1476/2938] 10.222.0.47 () {58 vars in 1153 bytes} [Thu Aug 4 05:20:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 05:20:57.597028 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fbff39-cad0-4227-ae1c-3955226f6d22 None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:57.608221 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fbff39-cad0-4227-ae1c-3955226f6d22 None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:57.613470 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a7d4d-788f-4abe-bce9-d3fdb3b7a44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:57.614125 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a7d4d-788f-4abe-bce9-d3fdb3b7a44b admin admin] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:57.614404 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a7d4d-788f-4abe-bce9-d3fdb3b7a44b admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:57.614676 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a7d4d-788f-4abe-bce9-d3fdb3b7a44b admin admin] PATH_INFO: `/v3/projects` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:57.615730 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a7d4d-788f-4abe-bce9-d3fdb3b7a44b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:57.619359 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a7d4d-788f-4abe-bce9-d3fdb3b7a44b admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:57.655941 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1463/2939] 10.222.0.47 () {60 vars in 1153 bytes} [Thu Aug 4 05:20:57 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:20:57.662047 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ae98a-13c9-448a-ab20-b6e23232981c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:57.676923 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ae98a-13c9-448a-ab20-b6e23232981c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:57.692602 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:57.755820 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb7e7e-d536-4e51-9657-f71df15e0a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:57.756486 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb7e7e-d536-4e51-9657-f71df15e0a5e admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:57.756845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb7e7e-d536-4e51-9657-f71df15e0a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:57.757165 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb7e7e-d536-4e51-9657-f71df15e0a5e admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:57.757911 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb7e7e-d536-4e51-9657-f71df15e0a5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:57.760374 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb7e7e-d536-4e51-9657-f71df15e0a5e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:57.789354 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1477/2940] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Aug 4 05:20:57 2022] POST /identity/v3/users => generated 286 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 05:20:57.796085 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a7d4d-788f-4abe-bce9-d3fdb3b7a44b admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:57.814072 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a7d4d-788f-4abe-bce9-d3fdb3b7a44b admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:57.819963 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937c0b6-010a-4613-b745-a33a379d2528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:57.820865 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937c0b6-010a-4613-b745-a33a379d2528 admin admin] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:57.821305 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937c0b6-010a-4613-b745-a33a379d2528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:57.821827 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937c0b6-010a-4613-b745-a33a379d2528 admin admin] PATH_INFO: `/v3/roles` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:57.823101 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937c0b6-010a-4613-b745-a33a379d2528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:57.826438 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937c0b6-010a-4613-b745-a33a379d2528 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:57.842590 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1464/2941] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:20:57.848054 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb7e7e-d536-4e51-9657-f71df15e0a5e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:57.864176 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb7e7e-d536-4e51-9657-f71df15e0a5e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:57.872004 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156869d1-caa9-4574-aae5-89f640b70e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:57.873653 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156869d1-caa9-4574-aae5-89f640b70e7f admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:57.874013 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156869d1-caa9-4574-aae5-89f640b70e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:57.874323 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156869d1-caa9-4574-aae5-89f640b70e7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:57.875505 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156869d1-caa9-4574-aae5-89f640b70e7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:57.878204 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156869d1-caa9-4574-aae5-89f640b70e7f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:57.892674 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1478/2942] 10.222.0.47 () {58 vars in 1122 bytes} [Thu Aug 4 05:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 05:20:57.897293 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937c0b6-010a-4613-b745-a33a379d2528 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:57.911308 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937c0b6-010a-4613-b745-a33a379d2528 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:57.916401 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9476f7-03ea-4005-8147-7853d8874c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:57.917220 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9476f7-03ea-4005-8147-7853d8874c45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:57.917574 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9476f7-03ea-4005-8147-7853d8874c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:57.917919 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9476f7-03ea-4005-8147-7853d8874c45 admin admin] PATH_INFO: `/v3/projects/0032dabb07a84bbe828e7f5d1368cade/users/155fa9d208aa476e94f901263dc5df41/roles/e733372bae6047be956ad703f83a480e` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:57.936206 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9476f7-03ea-4005-8147-7853d8874c45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0032dabb07a84bbe828e7f5d1368cade, user_id=155fa9d208aa476e94f901263dc5df41, role_id=e733372bae6047be956ad703f83a480e)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:57.938918 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9476f7-03ea-4005-8147-7853d8874c45 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:57.965774 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1465/2943] 10.222.0.47 () {60 vars in 1483 bytes} [Thu Aug 4 05:20:57 2022] PUT /identity/v3/projects/0032dabb07a84bbe828e7f5d1368cade/users/155fa9d208aa476e94f901263dc5df41/roles/e733372bae6047be956ad703f83a480e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 05:20:57.976285 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1853a55-d260-4119-98b4-e732a40b4085 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:57.976574 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1853a55-d260-4119-98b4-e732a40b4085 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:57.976846 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1853a55-d260-4119-98b4-e732a40b4085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:58.010659 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b1853a55-d260-4119-98b4-e732a40b4085 None None] MFA Rules not processed for user `155fa9d208aa476e94f901263dc5df41`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:20:58.012094 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b1853a55-d260-4119-98b4-e732a40b4085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:58.102291 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1479/2944] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:20:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 05:20:58.115191 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9476f7-03ea-4005-8147-7853d8874c45 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:58.117589 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-3a9476f7-03ea-4005-8147-7853d8874c45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:58.202155 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9476f7-03ea-4005-8147-7853d8874c45 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:58.206151 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b72a6ed-da3f-4882-87b9-7c76e8933b0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1d67b28dca49ca9cb0ff28cad62aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:58.206794 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b72a6ed-da3f-4882-87b9-7c76e8933b0c service cinder] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:58.207073 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b72a6ed-da3f-4882-87b9-7c76e8933b0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:58.207347 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b72a6ed-da3f-4882-87b9-7c76e8933b0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:58.217373 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b72a6ed-da3f-4882-87b9-7c76e8933b0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:58.218918 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b72a6ed-da3f-4882-87b9-7c76e8933b0c service cinder] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:58.229642 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1466/2945] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:20:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:20:58.267908 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1853a55-d260-4119-98b4-e732a40b4085 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:58.365279 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1853a55-d260-4119-98b4-e732a40b4085 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:58.370313 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed87ce-fc3e-4704-a99e-4ee109e06be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:58.371233 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed87ce-fc3e-4704-a99e-4ee109e06be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:58.371718 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed87ce-fc3e-4704-a99e-4ee109e06be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:58.372045 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed87ce-fc3e-4704-a99e-4ee109e06be5 admin admin] PATH_INFO: `/v3/users/155fa9d208aa476e94f901263dc5df41` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:58.375554 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed87ce-fc3e-4704-a99e-4ee109e06be5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=155fa9d208aa476e94f901263dc5df41)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:58.377703 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed87ce-fc3e-4704-a99e-4ee109e06be5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:58.407706 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:58.448404 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-95ed87ce-fc3e-4704-a99e-4ee109e06be5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '155fa9d208aa476e94f901263dc5df41'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:58.459463 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-95ed87ce-fc3e-4704-a99e-4ee109e06be5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '155fa9d208aa476e94f901263dc5df41'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:58.460662 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-95ed87ce-fc3e-4704-a99e-4ee109e06be5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '155fa9d208aa476e94f901263dc5df41'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:58.481036 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-95ed87ce-fc3e-4704-a99e-4ee109e06be5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '155fa9d208aa476e94f901263dc5df41'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:58.482011 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-95ed87ce-fc3e-4704-a99e-4ee109e06be5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '155fa9d208aa476e94f901263dc5df41'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:58.501505 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1480/2946] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:20:58 2022] DELETE /identity/v3/users/155fa9d208aa476e94f901263dc5df41 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:20:58.507893 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86d815-1567-416f-b3cd-1bae6afa2fce None None] REQUEST_METHOD: `POST` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:58.508271 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86d815-1567-416f-b3cd-1bae6afa2fce None None] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:58.508636 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86d815-1567-416f-b3cd-1bae6afa2fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:58.520537 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:20:58.539964 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.auth.core [None req-cf86d815-1567-416f-b3cd-1bae6afa2fce None None] MFA Rules not processed for user `fa79725fac2c4a528df2b060ede4922c`. Rule list: `[]` (Enabled: `True`). {{(pid=80403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 05:20:58.542996 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-cf86d815-1567-416f-b3cd-1bae6afa2fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:58.632261 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1467/2947] 10.222.0.47 () {58 vars in 958 bytes} [Thu Aug 4 05:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 05:20:58.643592 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed87ce-fc3e-4704-a99e-4ee109e06be5 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:58.645160 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-95ed87ce-fc3e-4704-a99e-4ee109e06be5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:58.736350 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed87ce-fc3e-4704-a99e-4ee109e06be5 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:58.740774 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482979d-91ed-48fb-8a69-f0404d5914cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:58.741536 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482979d-91ed-48fb-8a69-f0404d5914cd service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:58.741941 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482979d-91ed-48fb-8a69-f0404d5914cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:58.742351 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482979d-91ed-48fb-8a69-f0404d5914cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:58.759011 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482979d-91ed-48fb-8a69-f0404d5914cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:58.761934 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482979d-91ed-48fb-8a69-f0404d5914cd service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:58.775034 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1481/2948] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:20:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 05:20:59.173181 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86d815-1567-416f-b3cd-1bae6afa2fce None None] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:20:59.174608 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-cf86d815-1567-416f-b3cd-1bae6afa2fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:20:59.188498 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86d815-1567-416f-b3cd-1bae6afa2fce None None] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:20:59.193623 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121e80f-1212-4c2c-a565-41a068d11bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:20:59.194956 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121e80f-1212-4c2c-a565-41a068d11bf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:20:59.195348 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121e80f-1212-4c2c-a565-41a068d11bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:20:59.195949 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121e80f-1212-4c2c-a565-41a068d11bf4 admin admin] PATH_INFO: `/v3/projects/0032dabb07a84bbe828e7f5d1368cade` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:20:59.197801 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121e80f-1212-4c2c-a565-41a068d11bf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0032dabb07a84bbe828e7f5d1368cade)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:20:59.200659 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121e80f-1212-4c2c-a565-41a068d11bf4 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:20:59.272886 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a121e80f-1212-4c2c-a565-41a068d11bf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0032dabb07a84bbe828e7f5d1368cade', 'request_id': 'req-a121e80f-1212-4c2c-a565-41a068d11bf4'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:59.283219 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a121e80f-1212-4c2c-a565-41a068d11bf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0032dabb07a84bbe828e7f5d1368cade', 'request_id': 'req-a121e80f-1212-4c2c-a565-41a068d11bf4'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:59.291646 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a121e80f-1212-4c2c-a565-41a068d11bf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0032dabb07a84bbe828e7f5d1368cade', 'request_id': 'req-a121e80f-1212-4c2c-a565-41a068d11bf4'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:59.306068 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a121e80f-1212-4c2c-a565-41a068d11bf4 admin admin] The token cache is being invalidate because project 0032dabb07a84bbe828e7f5d1368cade was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:20:59.306384 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a121e80f-1212-4c2c-a565-41a068d11bf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:59.307052 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-a121e80f-1212-4c2c-a565-41a068d11bf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:20:59.308660 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1468/2949] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:20:59 2022] DELETE /identity/v3/projects/0032dabb07a84bbe828e7f5d1368cade => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:21:08.770089 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:DELETE=7,keystone:SELECT=46,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:21:09.302370 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=52,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:21:37.194563 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482979d-91ed-48fb-8a69-f0404d5914cd service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:21:37.196061 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a482979d-91ed-48fb-8a69-f0404d5914cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:21:37.209939 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482979d-91ed-48fb-8a69-f0404d5914cd service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:21:37.276768 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658586ac-854c-4e12-aa15-3d960a481e95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2a647f97fc4d369f562e8b5a4bf68c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:21:37.277425 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658586ac-854c-4e12-aa15-3d960a481e95 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:21:37.278078 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658586ac-854c-4e12-aa15-3d960a481e95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:21:37.278078 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658586ac-854c-4e12-aa15-3d960a481e95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:21:37.279923 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-658586ac-854c-4e12-aa15-3d960a481e95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:21:37.363970 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658586ac-854c-4e12-aa15-3d960a481e95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:21:37.365475 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658586ac-854c-4e12-aa15-3d960a481e95 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:21:37.377282 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1482/2950] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:21:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:21:45.557915 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121e80f-1212-4c2c-a565-41a068d11bf4 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:21:45.562985 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-a121e80f-1212-4c2c-a565-41a068d11bf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:21:45.593481 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121e80f-1212-4c2c-a565-41a068d11bf4 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:21:45.598940 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7bb95b-781d-4020-9994-5b2c99837012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:21:45.599667 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7bb95b-781d-4020-9994-5b2c99837012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:21:45.600014 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7bb95b-781d-4020-9994-5b2c99837012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:21:45.600334 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7bb95b-781d-4020-9994-5b2c99837012 admin admin] PATH_INFO: `/v3/users/30c0ff27c5a84ebd8615fb4608fc3882` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:21:45.601747 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7bb95b-781d-4020-9994-5b2c99837012 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30c0ff27c5a84ebd8615fb4608fc3882)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:21:45.603875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7bb95b-781d-4020-9994-5b2c99837012 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:21:45.660267 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-db7bb95b-781d-4020-9994-5b2c99837012 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30c0ff27c5a84ebd8615fb4608fc3882'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:21:45.660965 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-db7bb95b-781d-4020-9994-5b2c99837012 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30c0ff27c5a84ebd8615fb4608fc3882'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:21:45.675717 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-db7bb95b-781d-4020-9994-5b2c99837012 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30c0ff27c5a84ebd8615fb4608fc3882'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:21:45.686808 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-db7bb95b-781d-4020-9994-5b2c99837012 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30c0ff27c5a84ebd8615fb4608fc3882'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:21:45.687712 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-db7bb95b-781d-4020-9994-5b2c99837012 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30c0ff27c5a84ebd8615fb4608fc3882'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:21:45.711624 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1469/2951] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:21:45 2022] DELETE /identity/v3/users/30c0ff27c5a84ebd8615fb4608fc3882 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 05:21:45.941185 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658586ac-854c-4e12-aa15-3d960a481e95 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:21:45.962913 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658586ac-854c-4e12-aa15-3d960a481e95 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:21:46.046577 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b83d12-fcd2-49c3-b384-8be15ff48493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:21:46.047244 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b83d12-fcd2-49c3-b384-8be15ff48493 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:21:46.047571 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b83d12-fcd2-49c3-b384-8be15ff48493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:21:46.047845 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b83d12-fcd2-49c3-b384-8be15ff48493 admin admin] PATH_INFO: `/v3/projects/731637c0b96a4f1f8d9edb1af69e98cb` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:21:46.049080 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b83d12-fcd2-49c3-b384-8be15ff48493 admin admin] RBAC: Authorizing `identity:delete_project(project_id=731637c0b96a4f1f8d9edb1af69e98cb)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:21:46.051115 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b83d12-fcd2-49c3-b384-8be15ff48493 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:21:46.112781 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-50b83d12-fcd2-49c3-b384-8be15ff48493 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '731637c0b96a4f1f8d9edb1af69e98cb', 'request_id': 'req-50b83d12-fcd2-49c3-b384-8be15ff48493'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:21:46.115835 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:21:46.124182 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-50b83d12-fcd2-49c3-b384-8be15ff48493 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '731637c0b96a4f1f8d9edb1af69e98cb', 'request_id': 'req-50b83d12-fcd2-49c3-b384-8be15ff48493'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:21:46.132808 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-50b83d12-fcd2-49c3-b384-8be15ff48493 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '731637c0b96a4f1f8d9edb1af69e98cb', 'request_id': 'req-50b83d12-fcd2-49c3-b384-8be15ff48493'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:21:46.148219 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-50b83d12-fcd2-49c3-b384-8be15ff48493 admin admin] The token cache is being invalidate because project 731637c0b96a4f1f8d9edb1af69e98cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:21:46.148580 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-50b83d12-fcd2-49c3-b384-8be15ff48493 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:21:46.149467 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-50b83d12-fcd2-49c3-b384-8be15ff48493 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:21:46.151166 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1483/2952] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:21:45 2022] DELETE /identity/v3/projects/731637c0b96a4f1f8d9edb1af69e98cb => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:21:46.156666 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7bb95b-781d-4020-9994-5b2c99837012 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:21:46.159066 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-db7bb95b-781d-4020-9994-5b2c99837012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:21:46.174803 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7bb95b-781d-4020-9994-5b2c99837012 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:21:46.263573 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770131b6-0e66-47ef-b74d-b2248f401bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:21:46.264495 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770131b6-0e66-47ef-b74d-b2248f401bc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:21:46.264964 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770131b6-0e66-47ef-b74d-b2248f401bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:21:46.265330 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770131b6-0e66-47ef-b74d-b2248f401bc1 admin admin] PATH_INFO: `/v3/users/26ad3570fcf342b58eb6d4f6c3177591` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:21:46.267309 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770131b6-0e66-47ef-b74d-b2248f401bc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26ad3570fcf342b58eb6d4f6c3177591)` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:21:46.270933 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770131b6-0e66-47ef-b74d-b2248f401bc1 admin admin] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:21:46.343767 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-770131b6-0e66-47ef-b74d-b2248f401bc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26ad3570fcf342b58eb6d4f6c3177591'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:21:46.343767 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-770131b6-0e66-47ef-b74d-b2248f401bc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26ad3570fcf342b58eb6d4f6c3177591'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:21:46.358922 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-770131b6-0e66-47ef-b74d-b2248f401bc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26ad3570fcf342b58eb6d4f6c3177591'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:21:46.368587 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-770131b6-0e66-47ef-b74d-b2248f401bc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26ad3570fcf342b58eb6d4f6c3177591'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:21:46.369375 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.notifications [None req-770131b6-0e66-47ef-b74d-b2248f401bc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26ad3570fcf342b58eb6d4f6c3177591'} {{(pid=80403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:21:46.373875 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:21:46.388628 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1470/2953] 10.222.0.47 () {58 vars in 1224 bytes} [Thu Aug 4 05:21:46 2022] DELETE /identity/v3/users/26ad3570fcf342b58eb6d4f6c3177591 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:21:46.628534 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b83d12-fcd2-49c3-b384-8be15ff48493 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:21:46.630708 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-50b83d12-fcd2-49c3-b384-8be15ff48493 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:21:46.645815 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b83d12-fcd2-49c3-b384-8be15ff48493 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:21:46.737868 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86044f9b-29a5-4e70-868c-27436e80609c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa79725fac2c4a528df2b060ede4922c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '312c7b7a94774ca2ba3533f4d13a2124', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:21:46.738522 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86044f9b-29a5-4e70-868c-27436e80609c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:21:46.738800 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86044f9b-29a5-4e70-868c-27436e80609c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:21:46.739065 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86044f9b-29a5-4e70-868c-27436e80609c admin admin] PATH_INFO: `/v3/projects/4779f6b558784418a131a049703404a6` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:21:46.740289 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86044f9b-29a5-4e70-868c-27436e80609c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4779f6b558784418a131a049703404a6)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:21:46.742261 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86044f9b-29a5-4e70-868c-27436e80609c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:21:46.822196 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-86044f9b-29a5-4e70-868c-27436e80609c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4779f6b558784418a131a049703404a6', 'request_id': 'req-86044f9b-29a5-4e70-868c-27436e80609c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:21:46.834556 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-86044f9b-29a5-4e70-868c-27436e80609c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4779f6b558784418a131a049703404a6', 'request_id': 'req-86044f9b-29a5-4e70-868c-27436e80609c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:21:46.842239 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-86044f9b-29a5-4e70-868c-27436e80609c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4779f6b558784418a131a049703404a6', 'request_id': 'req-86044f9b-29a5-4e70-868c-27436e80609c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:21:46.857107 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-86044f9b-29a5-4e70-868c-27436e80609c admin admin] The token cache is being invalidate because project 4779f6b558784418a131a049703404a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 05:21:46.857489 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-86044f9b-29a5-4e70-868c-27436e80609c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:21:46.858397 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-86044f9b-29a5-4e70-868c-27436e80609c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 05:21:46.860280 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1484/2954] 10.222.0.47 () {58 vars in 1233 bytes} [Thu Aug 4 05:21:46 2022] DELETE /identity/v3/projects/4779f6b558784418a131a049703404a6 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 05:21:56.387610 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=4 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:21:56.852230 dv-cinder-851279-5-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:DELETE=10,keystone:SELECT=57,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 05:22:37.591386 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770131b6-0e66-47ef-b74d-b2248f401bc1 admin admin] Authenticating user token {{(pid=80403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 05:22:37.593124 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-770131b6-0e66-47ef-b74d-b2248f401bc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:22:37.683426 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770131b6-0e66-47ef-b74d-b2248f401bc1 admin admin] Validating token access rules against request {{(pid=80403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 05:22:37.766100 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f8f9b-35f0-4d5a-9c66-1e4710cb0898 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350f72f740f749f68175c1c4e8ae9d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f37a5f3140ec431da8ea14568654114a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 05:22:37.766784 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f8f9b-35f0-4d5a-9c66-1e4710cb0898 service placement] REQUEST_METHOD: `GET` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 05:22:37.767953 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f8f9b-35f0-4d5a-9c66-1e4710cb0898 service placement] SCRIPT_NAME: `/identity` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 05:22:37.768317 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f8f9b-35f0-4d5a-9c66-1e4710cb0898 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 05:22:37.770437 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.fernet_utils [None req-803f8f9b-35f0-4d5a-9c66-1e4710cb0898 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 05:22:37.802854 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f8f9b-35f0-4d5a-9c66-1e4710cb0898 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 05:22:37.805804 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f8f9b-35f0-4d5a-9c66-1e4710cb0898 service placement] RBAC: Authorization granted {{(pid=80403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 05:22:37.820176 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: [pid: 80403|app: 0|req: 1471/2955] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Aug 4 05:22:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 05:22:47.815017 dv-cinder-851279-5-iscsi devstack@keystone.service[80403]: DEBUG dbcounter [-] [80403] Writing DB stats keystone:SELECT=58 {{(pid=80403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}